Loss function
The score training uses to measure how wrong a model's guess was. Training tries to make this number smaller.
The fuller definition
A loss function measures how far a model's output is from a training objective. An optimizer uses this signal to guide parameter updates.
- ExamplesInputs paired with the expected output.
- PredictionThe model produces an output.
- LossHow far off it was, as a number.
- UpdateParameters nudged to reduce the loss; repeat.
Training ends at the knowledge cutoff. Whatever the examples contained, including their gaps and biases, is what the model learned.
A concrete example
Predicting a low probability for the target token produces a training penalty under a next-token objective.
The distinction that matters
A lower training loss does not alone prove better performance on your real workload.
Related terms
Training
Adjusting a model with examples and a learning rule. The aim is to improve how it handles tasks.
Training & generationGradient
The direction to nudge the model's numbers so its mistakes get smaller. Training takes many small steps in that direction.
Training & generationDataset
A collection of examples used to train or test a model.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.