Perplexity (metric)
Also: perplexity metric, perplexity score
A score for how surprised a model is by a piece of text. Lower is better, but it does not measure accuracy on your task.
The fuller definition
Perplexity is a measure of how well a language model predicts a text sample. Lower values mean the text was less surprising to the model. It is used to compare models and quantization levels.
Idealized value storage only, in decimal GB. Metadata, caches, mixed precision, and runtime overhead add to actual requirements. This is not a GPU purchasing calculator.
A concrete example
A quantized model shows slightly higher perplexity on a test corpus than the full-precision version.
The distinction that matters
Not to be confused with Perplexity the search product. Lower perplexity does not mean more accurate answers on your task.
Related terms
Evaluation (eval)
Testing an AI tool on a set of tasks to see how well it actually does.
Hardware & hostingQuantization
Storing model numbers with less detail. This can save memory. The effect on quality depends on the model and the task.
Training & generationLoss function
The score training uses to measure how wrong a model's guess was. Training tries to make this number smaller.
Where it is taught
Run a local model and verify the actual setupSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.