Batch processing
Also: batching
A group of tasks sent or run together. Some services charge less if the tasks can wait.
The fuller definition
Batch processing handles multiple items together rather than one independent operation at a time. In model serving, batching can share computational work across requests.
- Queue
Several inputs
- Batch
Grouped processing
- Results
Track each input separately
A simplified relationship, not a screenshot or a complete design.
A concrete example
A document pipeline queues a set of extraction requests for processing together.
The distinction that matters
Better throughput can come with waiting time, memory overhead, or different completion semantics.
Related terms
Where it is taught
APIs and JSON: connect a model to a small programSource and further reading
MDN · Web technology glossaryThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.