Test set
Also: test data, held-out set
Examples held back from training so you can check how the model does on things it has not seen.
The fuller definition
A test set contains examples held apart from model fitting or prompt development to estimate performance on unseen material.
- ExamplesInputs paired with the expected output.
- PredictionThe model produces an output.
- LossHow far off it was, as a number.
- UpdateParameters nudged to reduce the loss; repeat.
Training ends at the knowledge cutoff. Whatever the examples contained, including their gaps and biases, is what the model learned.
A concrete example
After tuning a chronology prompt on five records, assess it on separate records with similar difficulty.
The distinction that matters
Repeatedly revising against the same test answers turns that set into development material.
Related terms
Dataset
A collection of examples used to train or test a model.
Reliability & securityEvaluation (eval)
Testing an AI tool on a set of tasks to see how well it actually does.
Training & generationOverfitting
When a model fits its training examples too closely. It may then do poorly on new examples.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.