batching

PyTorch/XLA integration with JetStream (https://github.com/google/JetStream) for LLM inference"

Document level Attitude and Relation Extraction toolkit (AREkit) for sampling and processing large text collections with ML and for ML

BRRRRRRRRRRRRRRRRRRRRRR

Batch-scheduler framework for controlling execution in a packet-processing pipeline based on strict service-level objectives

FastDynamicBatcher is a library for batching inputs across requests to accelerate machine learning workloads

NASA Harmony service that groups together files into batches for concatenation

Batching with python asyncio

TurboBatch accelerates transformer inference by up to 10.2x with dynamic batching. It's lightweight, HuggingFace-compatible, and ideal for real-time NLP tasks.

Production-grade self-hosted LLM inference server optimized for GPU batching, parallel request scheduling, and high-throughput LAN deployment.

Batch any Python iterator by count, elapsed time, or both. Zero dependencies.

to save cost latency

High-performance embedding inference service with dynamic batching, FastAPI, and GPU optimization.

simulation of Bucket brigade in production lines.

Official Python SDK for LogTide - Production-ready logging with automatic batching, circuit breaker, distributed

SHIRT: SHIRT Handles Intense Renaming Transformations - A command-line tool for renaming and encoding files and directories.

Event-driven benchmark of adaptive batch composition policies for LLM serving, measuring how prefill and decode interference affects TTFT, TPOT, and throughput under different memory pressure regimes.

A lightweight async batching queue for Python

Migrates public entity financial data from EDGAR zip into AWS S3 bucket

Plan offline embedding batches against configurable item and token capacity limits.

Benchmark of sequence packing and length-aware batching for variable-length LLM requests, measuring compute waste from padding and the latency-efficiency trade-off between offline and online grouping policies.