Fine-tuning teaches a model behaviour from examples. It suits format, tone and narrow tasks, not fast changing facts. Start with evals and prompting, use adapters to keep costs down, and test for forgetting and safety changes. Short answers follow, with links to detail.
Should I fine-tune at all?
Only if evals show prompting and retrieval are not enough. The OpenAI guide says prompt engineering may be enough, and lists tuning for more examples than fit in context, shorter prompts, sensitive data and smaller models. Detail: when to fine-tune or just prompt.
Fine-tuning or RAG?
RAG for facts that change or need citations. Tuning for behaviour and format. Many systems use both. Detail: fine-tuning versus RAG.
How much data?
Often hundreds to a few thousand clean examples. The LIMA paper used 1,000 curated pairs on a 65B base. Detail: how much data you need.
What is LoRA?
A method that freezes the base model and trains small added matrices. The paper reports up to 10,000 times fewer trainable parameters and 3 times less GPU memory in its GPT-3 175B comparison. Detail: what LoRA is.
What is QLoRA?
LoRA on top of a frozen 4-bit model, reported to allow tuning a 65B model on one 48 GB GPU. Detail: QLoRA explained.
What is the difference between SFT and DPO?
SFT trains on ideal answers. DPO trains on better and worse pairs. Detail: SFT, DPO and reinforcement fine-tuning.
Will tuning make the model forget things?
It can. Keep retention evals. Detail: catastrophic forgetting.
Does tuning affect safety?
A 2023 paper found it can weaken safety alignment, even with benign data. Detail: fine-tuning and safety.
Can I train on model generated data?
Yes with filtering and mixing, but beware model collapse. Detail: synthetic data uses and risks.
What does it cost?
Data and experiments usually exceed the final run. Detail: budgeting guide.
Can I trust benchmark scores?
Use them to shortlist, then run your own evals. Detail: reading benchmark scores.
A short order of operations
If you remember nothing else, remember the order. First, write the eval set and score your best prompt. Second, add retrieval if the gap is missing knowledge. Third, tune a small adapter if the gap is behaviour or format, using a few hundred clean, deduplicated examples with a held-out split. Fourth, rerun the target evals, the retention evals and a safety check. Fifth, record the base model version, data version and settings so the run can be repeated.
Skipping the first step is the most common mistake. Without a baseline, a tuned model that feels better may not be better, and you cannot tell what the training cost bought you.
Quick check
What is the first thing to build?
An eval set from real inputs, with a baseline score from your best prompt.
Which method is cheapest to try first?
A LoRA adapter on a small base model with a few hundred clean examples.
Sources
- OpenAI model optimization guide, fetched 19 October 2026.
- LIMA: Less Is More for Alignment, fetched 19 October 2026.
- LoRA, fetched 19 October 2026.
- QLoRA, fetched 19 October 2026.
- Fine-tuning Aligned Language Models Compromises Safety, fetched 19 October 2026.
Last reviewed: 19 October 2026.
