VRAM — Video random-access memory
Also: GPU memory
The memory on the graphics card. A model has to fit in it to run quickly.
Running a model on your own machine means loading all of its numbers into memory, like luggage into an overhead bin. Quantization shrinks the luggage. The graphics card’s memory (VRAM) is the bin that matters for speed.
The fuller definition
Video random-access memory is memory available to a GPU for data and computation. In AI workloads it may hold model weights, caches, and intermediate results.
Go deeper
Idealized value storage only, in decimal GB. Metadata, caches, mixed precision, and runtime overhead add to actual requirements. This is not a GPU purchasing calculator.
A concrete example
A model's weight files fit on disk but exceed available GPU memory once runtime overhead is included.
The distinction that matters
Two GPUs' memory is not automatically one transparent pool; the runtime must support distributing the workload.
Related terms
GPU — Graphics processing unit
The chip that handles graphics, and also the maths AI needs. It is what makes AI fast.
Hardware & hostingKV cache — Key-value cache
Stored results from part of a model's earlier work. Reusing them can speed up the next step. This cache also takes memory.
Hardware & hostingQuantization
Storing model numbers with less detail. This can save memory. The effect on quality depends on the model and the task.
Where it is taught
Understand the hardware a local model needsSource and further reading
Hugging Face · Model inference and quantizationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.