Name an exact model version in configuration rather than a floating alias, keep a small evaluation set of real examples, and promote a newer version only after it passes. Keep the previous version available so rollback is one config change. Check your provider's documentation for how versions are named and retired.
Why pin
A floating alias can point to a different model over time. The behavior of your feature, including its refusals, formats, and mistakes, can shift without any change in your code. Pinning makes the model a reviewed dependency, like a library version in a lockfile.
Semantic Versioning for libraries works because a number promises something about compatibility. Model versions carry no such promise, so your own tests have to supply it.
Steps
- Find the naming scheme. Read how the provider names versions and aliases, and how it announces retirements. Do not assume.
- Pin in config. Store the exact identifier in one place, read by every service. Avoid hard coding it in prompts or code.
- Build an evaluation set. Collect real, representative inputs, including awkward and adversarial ones, with the outputs you consider acceptable. A few dozen well chosen cases beat a large unreviewed set.
- Define pass criteria. Decide before testing what counts as a regression: format errors, refused tasks, unsafe outputs, cost per request.
- Run the candidate. Test the new version against the set, and compare to the pinned one.
- Promote gradually. Move a small share of traffic, watch the metrics, then widen.
- Keep rollback ready. The old identifier stays in config until the new version has run cleanly for an agreed period.
What to log
| Item | Reason |
|---|---|
| Model identifier on every request | Trace behavior to a version |
| Prompt template version | Separate prompt changes from model changes |
| Evaluation results per version | Evidence for the decision |
| Approver and date | Change record |
Plan for retirement
Providers retire versions. Put a recurring check on the roadmap: which pinned versions have announced end dates, and when must we move? Treat it like the end of security support for a framework, and start early enough to test.
Treat the change like a release
A model change deserves the same gate as code: owner, tests, record. Customers who ask how AI features are controlled will want to hear exactly this.
Keep humans in the loop for the first runs
When a new version goes to a small share of traffic, have someone read a sample of real outputs each day. Metrics catch format errors and cost shifts, but a person notices tone and judgment problems that no check was written for.
Quick check
Is a small evaluation set enough?
It is enough to catch obvious regressions. Add cases whenever production shows a new failure.
Should the model change need approval?
Yes. Use the same sign off as other production changes, with test results attached.
Safeguard's policies, gates, and guardrails and its compliance evidence automation can record model change decisions, and the free scan is at gold.safeguard.sh/scan.
Last reviewed: 19 October 2026.
