Catastrophic forgetting is when a neural network trained on a new task loses performance on what it learned before. For teams fine-tuning language models, it shows up as a model that nails your new task but gets worse at general questions, formats or languages it handled before.
Where the term comes from
Researchers long assumed that networks trained on tasks one after another would overwrite earlier knowledge. A 2016 paper, Overcoming catastrophic forgetting in neural networks, proposed slowing learning on the weights most important to earlier tasks. The authors tested it on sequential MNIST tasks and on several Atari 2600 games learned in sequence, and reported that it scales and works effectively.
That method is one answer. The practical lesson for fine-tuning is simpler: training moves weights, and moved weights can break old behaviour.
What it looks like in practice
- A support model tuned on short replies stops writing long explanations when asked.
- A classifier tuned on English tickets degrades on other languages it used to handle.
- A model tuned to answer in a strict format refuses or fails on ordinary chat.
- Safety behaviour weakens. See how fine-tuning can weaken a model's safety training.
How to detect it
Keep two eval sets, not one:
- Target evals for the new task. These should go up.
- Retention evals for things you did not mean to change: general instructions, other formats, languages you support, refusal behaviour. These should stay flat.
Run both before and after every training run. Without the second set, you only see the win.
Ways to limit it
- Use adapters. Methods like LoRA freeze the base weights and train small additions, which limits how far the model can drift. It does not remove the risk entirely, so still run retention evals.
- Lower the learning rate and train fewer steps. Stop when target evals plateau.
- Mix in general data. Blend examples of the behaviours you want to keep with the new task data.
- Tune narrowly. Train for one clear task rather than a pile of unrelated ones.
- Keep the base model. Route non-target requests to the untuned model.
- Regularization methods. Research methods such as the one in the 2016 paper exist, but check library support before relying on them.
A routing alternative
If forgetting keeps hurting, do not force one model to do everything. Use a small tuned model for the narrow task and a general model for everything else, with a simple classifier or rule deciding which to call.
Document the trade
Write down what you tested, what moved, and what you accepted. A model card with retention results saves an argument later when someone finds a regression.
Quick check
Does LoRA prevent catastrophic forgetting?
It reduces drift because the base weights stay frozen, but behaviour can still change. Measure with retention evals.
How do I know if my model forgot something?
Run a fixed retention eval set before and after training and compare scores.
Sources
- Overcoming catastrophic forgetting in neural networks, fetched 19 October 2026.
Last reviewed: 19 October 2026.
