Low-rank adaptation (LoRA)
A cheap way to fine-tune: instead of changing all the model's numbers, you train a small add-on that sits beside them.
Fine-tuning changes how the model writes, not what it knows about this week’s facts. For facts, give it the documents instead.
The fuller definition
Low-rank adaptation trains small additional parameter matrices while generally keeping the original model weights frozen. It is a parameter-efficient fine-tuning method.
Go deeper
Supply instructions and examples each time. Nothing in the model changes.
Fetch current documents and put them in context. Facts can change daily.
Train the parameters on examples. Changes behaviour and style; poor at keeping facts current.
Most legal tasks need the first two. Fine-tuning is a project, with data, evaluation, and maintenance, and it is the wrong tool for facts that change.
A concrete example
A specialized adapter is trained for a formatting task and used with a compatible base model.
The distinction that matters
The adapter still requires training, suitable data, and evaluation; it is not just a prompt file.
Related terms
Fine-tuning
More training on a chosen set of examples. It changes a model to suit a task or style.
FoundationsParameter
One of the billions of adjustable numbers inside a model. Training sets them; using the model does not change them.
FoundationsModel weights
The full set of numbers that make up a trained model. Download the weights and you have the model.
Where it is taught
How models learn from examplesSource and further reading
Hugging Face · LoRAThe definition and example are written for this library. The link gives background, official documentation or legal authority as labelled. Legal scope depends on the jurisdiction and current text; product features depend on release and settings.