Model serving
Also: inference server
Running a model so that other programs can send it requests.
The fuller definition
Model serving makes a loaded model available to applications through an interface, usually managing requests, resources, and output delivery.
- Load
Model and configuration
- Serve
Accept requests
- Operate
Monitor errors and resources
A simplified relationship, not a screenshot or a complete design.
A concrete example
A local server exposes an API so a separate document application can request model responses.
The distinction that matters
Running a model once in a notebook differs from operating a reliable service for multiple users.
Related terms
Server
A computer that runs somewhere else and answers requests from your device.
APIs & softwareAPI — Application Programming Interface
A doorway that lets one program talk to another. Apps use an API to send your request to the model and get the answer back.
Reliability & securityObservability
Being able to see what an AI system is doing inside, so problems can be found and explained.
Where it is taught
Understand the hardware a local model needsSource and further reading
Hugging Face · Model inference and quantizationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.