Local model
Also: local AI, local inference
A model that runs on your own computer, so your documents never leave it.
Running a model on your own machine means loading all of its numbers into memory, like luggage into an overhead bin. Quantization shrinks the luggage. The graphics card’s memory (VRAM) is the bin that matters for speed.
The fuller definition
A local model runs inference on hardware under your control rather than requiring a remote provider for that computation.
Go deeper
- Your computer
- Model running on it
- Answer stays on it
Confidentiality depends on the machine and anything else connected to it.
- Your computer
- Provider's servers
- Storage, logs, possibly training
- Answer returned
Confidentiality depends on the contract, the account tier, and the settings in force.
Neither route is safe by default. The question is the same for both: who can see the material, where is it kept, and for how long.
A concrete example
A desktop loads compatible weights and answers through a local application.
The distinction that matters
Model downloads, telemetry, updates, and connected tools may still use the network; inspect the complete setup.
Related terms
Ollama
Software for downloading and running AI models. It can run local models, but you still need to check which model and service you chose.
FoundationsModel weights
The full set of numbers that make up a trained model. Download the weights and you have the model.
Hardware & hostingCloud inference
Running a model on a remote server. Your input travels to that server, so check its data terms before use.
Where it is taught
Understand the hardware a local model needsSource and further reading
Hugging Face · Model inference and quantizationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.