Gradient
Also: gradients, gradient descent, optimizer, optimiser
The direction to nudge the model's numbers so its mistakes get smaller. Training takes many small steps in that direction.
The fuller definition
A gradient describes how a quantity such as loss changes with respect to parameters. Gradient-based optimization uses that information to update parameters.
- ExamplesInputs paired with the expected output.
- PredictionThe model produces an output.
- LossHow far off it was, as a number.
- UpdateParameters nudged to reduce the loss; repeat.
Training ends at the knowledge cutoff. Whatever the examples contained, including their gaps and biases, is what the model learned.
A concrete example
Training changes a weight in a direction expected to reduce loss for a small step.
The distinction that matters
A gradient is a numerical calculation, not an explanation in ordinary language.
Related terms
Loss function
The score training uses to measure how wrong a model's guess was. Training tries to make this number smaller.
Training & generationTraining
Adjusting a model with examples and a learning rule. The aim is to improve how it handles tasks.
FoundationsParameter
One of the billions of adjustable numbers inside a model. Training sets them; using the model does not change them.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.