GPU — Graphics processing unit
Also: GPUs
The chip that handles graphics, and also the maths AI needs. It is what makes AI fast.
Running a model on your own machine means loading all of its numbers into memory, like luggage into an overhead bin. Quantization shrinks the luggage. The graphics card’s memory (VRAM) is the bin that matters for speed.
The fuller definition
A Graphics Processing Unit is a processor designed for highly parallel computation, used for graphics and many AI calculations.
Go deeper
- Many values
Tensor data
- Parallel compute
GPU operations
- Result
Model calculations
A simplified relationship, not a screenshot or a complete design.
A concrete example
A model runtime uses a GPU to perform large numbers of tensor operations in parallel.
The distinction that matters
Owning a GPU does not guarantee compatibility with a particular AI runtime or that a model fits its memory.
Related terms
VRAM — Video random-access memory
The memory on the graphics card. A model has to fit in it to run quickly.
Hardware & hostingCPU — Central processing unit
The main brain of a computer. It can run AI models, but slowly.
FoundationsTensor
A grid of numbers (a table, or a stack of tables). Everything inside a model is stored and calculated as tensors.
Where it is taught
Understand the hardware a local model needsSource and further reading
Hugging Face · Model inference and quantizationThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.