Bit depth and precision
Also: numerical precision, storage precision, lower precision, mixed precision, FP16, FP32, BF16, INT8, 4-bit, 8-bit, 16-bit
How much detail each number in a model is stored with. Fewer bits means a smaller, faster, slightly rougher model.
The fuller definition
Numerical precision describes how values are represented with bits. Formats differ in range and precision, not just size; AI systems may mix formats for weights and computation.
Idealized value storage only, in decimal GB. Metadata, caches, mixed precision, and runtime overhead add to actual requirements. This is not a GPU purchasing calculator.
A concrete example
An idealized eight-billion-value array uses 16 billion bytes at 16 bits per value, before metadata and other overhead.
The distinction that matters
File-size arithmetic is a lower-level estimate, not a complete hardware requirement.
Related terms
Quantization
Storing model numbers with less detail. This can save memory. The effect on quality depends on the model and the task.
Hardware & hostingVRAM — Video random-access memory
The memory on the graphics card. A model has to fit in it to run quickly.
FoundationsParameter
One of the billions of adjustable numbers inside a model. Training sets them; using the model does not change them.
Where it is taught
Understand the hardware a local model needsSource and further reading
Hugging Face · Model inference and quantizationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.