GGUF
The file format most local AI apps use for downloaded models.
The fuller definition
GGUF is a file format used in the llama.cpp ecosystem to package model tensors and metadata for compatible inference runtimes.
- Metadata
Architecture and configuration
- Tensors
Model values
- Runtime
Read a supported file
A simplified relationship, not a screenshot or a complete design.
A concrete example
A downloaded GGUF file identifies a model variant and may contain quantized tensors.
The distinction that matters
A file extension does not guarantee a runtime supports that architecture or that its contents are trustworthy.
Related terms
llama.cpp
Software that runs AI models on your machine. Many local chat apps use it behind the scenes.
Hardware & hostingQuantization
Storing model numbers with less detail. This can save memory. The effect on quality depends on the model and the task.
FoundationsModel weights
The full set of numbers that make up a trained model. Download the weights and you have the model.
Where it is taught
Understand the hardware a local model needsSource and further reading
ggml-org · GGUF specificationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.