Latency
The wait between sending a request and getting the answer back.
The fuller definition
Latency is elapsed time between a request and a defined response point, such as the first output token or the complete answer.
- Request
Start timer
- First token
Initial response delay
- Completion
Full result received
A simplified relationship, not a screenshot or a complete design.
A concrete example
One system begins answering quickly but takes longer to finish a large response.
The distinction that matters
Time to first token and time to completion measure different aspects of responsiveness.
Related terms
Throughput
How much work a system can get through in a given time.
FoundationsInference
The act of running a trained model to get an answer. In law the word means something else: a conclusion drawn from evidence.
APIs & softwareStreaming
Showing the answer word by word as it is written, instead of waiting for the whole thing.
Where it is taught
APIs and JSON: connect a model to a small programSource and further reading
MDN · Web technology glossaryThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.