Top-p sampling
Also: nucleus sampling
A setting that narrows the next-token choices. It keeps a group with a set share of the total chance. Lower values usually narrow that group.
Always picks the most likely next word. Ask twice, get the same answer twice.
Sometimes picks a less likely word. Ask twice, get two different answers.
Neither setting makes the answer more true. Low temperature makes mistakes repeatable; high temperature makes them varied.
The fuller definition
Top-p sampling restricts candidate tokens to a set whose cumulative probability reaches a chosen threshold, then samples within that set.
Go deeper
Invented probabilities. Always picking the top candidate gives the same wording every run; sampling sometimes picks a lower one. Neither setting checks whether the sentence is true.
A concrete example
With probabilities 0.5, 0.3, and 0.2, a threshold of 0.8 can retain the first two candidates.
The distinction that matters
The threshold controls selection, not the probability that the final answer is true.
Related terms
Sampling
How the model picks the next word from its list of likely candidates. It can always take the top one, or roll the dice among the top few.
Training & generationTemperature
A setting that affects how varied the output may be. Lower values often make it less varied. They do not ensure the same answer each time.
FoundationsToken
A piece of text, often part of a word, that the model actually works with. Models read and write tokens, not whole words.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.