Overfitting
When a model fits its training examples too closely. It may then do poorly on new examples.
The fuller definition
Overfitting occurs when a system fits the particularities of its development data so closely that its performance does not generalize well to new examples.
- ExamplesInputs paired with the expected output.
- PredictionThe model produces an output.
- LossHow far off it was, as a number.
- UpdateParameters nudged to reduce the loss; repeat.
Training ends at the knowledge cutoff. Whatever the examples contained, including their gaps and biases, is what the model learned.
A concrete example
A prompt handles every practiced email but fails when the same issue is phrased differently.
The distinction that matters
Excellent results on familiar examples do not establish dependable performance on future records.
Related terms
Test set
Examples held back from training so you can check how the model does on things it has not seen.
Training & generationDataset
A collection of examples used to train or test a model.
Reliability & securityEvaluation (eval)
Testing an AI tool on a set of tasks to see how well it actually does.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.