Base model and instruct model
Also: chat model, instruction-tuned model, instruction tuning
A base model just continues text. An instruct model has been trained to follow instructions and chat.
“The judge entered the ___”
It picks a word, adds it to the sentence, and guesses again, thousands of times a second. Everything it writes, from a poem to a legal memo, is built this way. That is why it can be fluent and wrong at the same time: it is predicting what sounds right, not checking what is true.
The fuller definition
A base model is the result of pretraining: it continues text. An instruct or chat model is further trained to follow instructions and hold conversations, usually with safety training added.
Go deeper
Supply instructions and examples each time. Nothing in the model changes.
Fetch current documents and put them in context. Facts can change daily.
Train the parameters on examples. Changes behaviour and style; poor at keeping facts current.
Most legal tasks need the first two. Fine-tuning is a project, with data, evaluation, and maintenance, and it is the wrong tool for facts that change.
A concrete example
Given the words Summarize this letter, a base model may continue with more letters. The instruct version produces a summary.
The distinction that matters
Most downloadable model families offer both. The suffix in a file name, such as Instruct, matters for behaviour.
Related terms
Pretraining
An early, broad stage of model training. The model learns patterns from a large set of data before later training for some uses.
Training & generationFine-tuning
More training on a chosen set of examples. It changes a model to suit a task or style.
Training & generationReinforcement learning from human feedback (RLHF)
Training that uses people's ratings of model answers. It helps steer the model toward replies people prefer.
Where it is taught
Prompting, retrieval, or fine-tuning?Source and further reading
Hugging Face · LLM courseThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.