Diagnose what is missing
If the model does not have the operative agreement, improve the source route. If it has the agreement but the task is vague, improve the brief. If a stable, repeated behavior remains poor despite good instructions and examples, training may be worth investigating. These interventions address different problems.
Fine-tuning is additional training. LoRA is a method that trains relatively small update matrices while keeping original model weights frozen. It can reduce the number of parameters that need training. Quantization concerns numerical representation and memory; it is not a mechanism for adding your matter documents to an application's search. [source] [source]
Use a decision table
| Observed problem | First thing to investigate |
|---|---|
| Wrong or missing source | Ingestion, versions, retrieval, or access |
| Inconsistent output format | Clearer schema, examples, and validation |
| Repeated style issue | Specific instructions and accepted examples |
| Narrow classification task at scale | Measured baseline; consider training if justified |
| Slow local model | Placement, model size, context, runtime, and workload |
The table proposes an order of investigation. It does not establish that prompting can solve every behavior problem or that training is always needed for classification.
Work a style example
Suppose the model routinely turns qualified statements into categorical ones. Start with a rule and three contrasting examples. Test on fresh sentences. If the failure persists across a large, stable workload, consider whether a curated training set could improve behavior.
Before training, write a baseline evaluation. Separate examples used for development from held-out evaluation. If accepted and rejected drafts contain confidential facts, do not assume they are suitable training material simply because the original task was authorized. Determine the actual permission and architecture.
Understand the continuing maintenance
A trained adaptation can reproduce an unwanted pattern as readily as a useful one. Label quality, dataset balance, and version tracking matter. If the task changes, old training may encode assumptions you no longer want. Keep the baseline and compare against both old and new cases.
For changing matter facts, a source system with identifiable versions and corrections is usually easier to audit than trying to make model weights serve as the authoritative record. The practical question is how an error can be found, corrected, and traced through later outputs.
Try it: the wrong intervention
A system misses an exception because the PDF extraction omitted page 46. Someone proposes fine-tuning on more examples of careful legal analysis. What should happen first?
Check your understanding
Repair extraction and coverage, then rerun the task with the missing exception present. Training on better reasoning examples does not restore text the system never received. Once the source route works, assess whether an interpretation failure remains.
A useful stopping rule
Do not fine-tune merely because the technique is available. State the target behavior, the measured baseline, the lawful and suitable dataset, the evaluation method, and the cost of maintaining it. If those are undefined, you do not yet have a training project; you have a problem that still needs diagnosis.