Models
Fine-tuning
Fine-tuning continues training a base model on your own examples so it adopts a format, tone or behaviour without being told each time.
It pays back by shortening the prompt: instructions, examples and format specifications move into the weights. If it does not shorten the prompt, it is unlikely to pay for itself, since fine-tuned inference usually costs more per token.
Try the cheaper options first — a better prompt, few-shot examples, then retrieval. Fine-tuning is the slowest to iterate on because every change means another training run.
In practice
Fine-tuning teaches format and behaviour; retrieval supplies facts. If the answer would be different next month, it belongs in retrieval — a fine-tune bakes the knowledge in at training time and updating it means training again. The common expensive mistake is fine-tuning to fix a problem that was a prompt ambiguity.
Common questions
When should I fine-tune instead of prompting?
When you need consistent format or behaviour across high volume and the prompt to achieve it has become long. Not for knowledge that changes — that belongs in retrieval, where updating it does not mean training again.
How many examples does fine-tuning need?
Providers commonly suggest a minimum in the low hundreds, but quality dominates quantity: a few hundred carefully checked examples outperform thousands of inconsistent ones, because the model learns the inconsistency too.
Measure it
- Fine-Tuning Cost and Payback EstimatorTraining cost against the prompt tokens it removes, with payback in months.
- JSONL Validator for Fine-Tuning DatasetsLine-numbered parse errors plus the structural checks uploads actually fail on.
- Dataset Deduplicator for Fine-TuningCatches duplicates that differ only in case, spacing or punctuation.