Keep prompts in the build by default. A prompt shipped with a release goes through review, testing and rollback like code. Loading prompts from a database or remote store at runtime is reasonable when you add controls: restricted write access, change approval, versioning and an audit trail.
The two approaches
In the build means the template is a file in your repository, packaged into the release. Changing it means a pull request, a pipeline run and a deploy.
At runtime means the application fetches the template from a database, a configuration service or a hosted prompt tool each time, or on a cache refresh. Changing it means editing a record.
Why teams want runtime loading
Speed is the honest reason. Non engineers can tune wording without waiting for a release, and experiments are cheap. Those are real benefits. The cost is that the edit path now bypasses everything your pipeline normally does.
What you give up
- Review. A record edit has no pull request unless you build one.
- Testing. Nothing runs your sample inputs before the change goes live.
- Rollback. Without version history on the record, you cannot restore the old text with confidence.
- Blast radius. One edit can change behavior across every instance at once.
- Access creep. Database write access tends to be broader than repository write access.
If the prompt tells a model which tools to use or what data it may reveal, a runtime edit is closer to a production code change than a content change.
Comparison
| Question | In the build | Runtime store |
|---|---|---|
| Who can change it | Anyone who can merge | Anyone with write access to the store |
| Review by default | Yes, via pull request | No, must be added |
| Tests before live | Pipeline runs them | Must be added |
| Rollback | Redeploy previous release | Depends on version history |
| Speed of change | Slower | Fast |
| Audit trail | Git history | Depends on the store |
If you choose runtime loading
Add these before launch, not after an incident.
- Limit write access to a small named group, and require sign in with individual accounts.
- Keep every version, with author and timestamp, and never edit in place.
- Require a second person to approve changes to prompts that control tools or data access.
- Run your sample test inputs against a candidate version before promoting it.
- Log the version used on each model call.
- Mirror the final text into the repository so the code history stays complete.
- Provide a one step way to return to the previous version.
A middle path
Many teams split prompts by risk. Low risk text, such as tone or formatting hints for an internal tool, can live in a runtime store. High risk text, such as instructions that grant tool access or define refusals, stays in the build. Write the split down so nobody has to guess.
Decision guide
- Does the prompt influence tool calls, data access or customer facing claims? Keep it in the build.
- Do you need changes within minutes during an incident? Allow runtime loading with a fast rollback and an audit trail.
- Is the store shared with other teams or tools? Treat its write access as a risk in its own right.
For the version control side of this, see how to version and review prompt templates in Git. For the supply chain framing, see are prompt templates part of your software supply chain.
Quick check
Is a hosted prompt tool safer than my own database?
Not automatically. What matters is who can write, whether versions are kept and whether changes need approval. Ask the same questions of any store.
Can I cache prompts and still roll back fast?
Yes, if the cache has a short expiry or a way to flush it, and you log the version served.
Last reviewed: 19 October 2026.
