Once a quarter, walk the prompt and dataset inventory and check five things: the owner still exists, the origin is still known, the content still matches the recorded version, the license and risk tier still hold, and the tests still run. Retire what nobody uses. One hour per product team is a reasonable start.
Why a recurring review
Inventories decay. People change teams, prompts gain tools, datasets get refreshed by a script nobody watched, and a license changes. A short, regular review catches drift before an incident or an audit does.
The checklist
Ownership
- Every item has a named owner or team that still exists.
- Escalation contacts are reachable.
Origin and change
- Each prompt's text matches the version in the repository and the version in production.
- Each dataset's current hash matches the recorded hash. Investigate any mismatch before use.
- Any text or data adopted from outside since last quarter has a recorded source.
Risk tier
- Prompts that gained a tool, a new data source or customer exposure have moved to a higher tier.
- Datasets that now feed customer facing models have been reviewed again.
Rights and sensitivity
- Licenses and terms are still on file and still cover the use. This is a record check, not legal advice.
- A sample of data was checked for personal or confidential content.
Testing
- The regression set still runs and still passes.
- At least one new case was added from a real issue this quarter.
Retirement
- Unused prompts and datasets are archived or deleted.
- Cached copies of retired data are removed.
Tracking table
| Item | Type | Owner | Last hash or version check | Tier | Action |
|---|---|---|---|---|---|
| ticket-summarizer | Prompt | Support platform | 2026-10-19 | Lower | None |
| support-corpus-2026 | Dataset | Data team | 2026-10-19 | Higher | Re check license |
Keep this in the repository, so each review is a dated commit.
Make drift visible between reviews
A quarter is a long time. Where you can, automate the cheap parts: a job that recomputes dataset hashes and alerts on mismatch, and a pipeline check that fails if a prompt file changes without a version bump. Quarterly review then focuses on judgment, not on finding differences.
Who attends
Keep it small: the owners, one security reviewer and someone from the team that consumes the output. Thirty minutes of review per product plus thirty of follow up is realistic.
What a good outcome looks like
By the end you should have a list of changes made, a list of items retired, and a list of open questions with names and dates. Any item that cannot be given an owner is a candidate for removal, because something nobody owns cannot be defended.
Where Safeguard fits
Safeguard's continuous monitoring and SBOM tracking keep the dependency side of an AI application under watch between reviews, and policies can hold changes at a pull request gate. Prompt and dataset review itself stays a human routine. A free scan is at https://gold.safeguard.sh/scan.
Try it yourself
Click through it at your own pace, or open it full screen. Watch how a change appears between scheduled checks, then think about which prompt or dataset checks could raise the same kind of alert.
Quick check
Is quarterly often enough?
For stable products, yes, paired with automatic hash and change checks. Review sooner after an incident or a major model change.
Who signs off the review?
The owner of each item, with security signing off on higher tier entries.
Last reviewed: 19 October 2026.
