For each model call, log identifiers that let you rebuild what happened: the prompt template version, the model version, the retrieved document IDs and the index snapshot, plus the dataset hashes behind the model. Be careful with raw text, which may hold personal data. Identifiers answer most triage questions without storing the content.
The question logs must answer
During an incident someone asks: why did the assistant say that? To answer you need to know which instructions it had, which model produced the output, and which content it saw. If any of the three is unknown, you are guessing.
What to log per request
| Field | Purpose |
|---|---|
| Request ID and time | Join across systems |
| Prompt template name and version | Which instructions were live |
| Model name and exact version | Provider changes can alter behavior |
| Retrieved document IDs and source | Which content entered the prompt |
| Index snapshot or version | Which state of the corpus was searched |
| Tool calls made, with arguments summary | What actions the model triggered |
| Output validation result | Whether a check rejected or altered the answer |
| User or tenant identifier | Scope the impact |
What to log per model or dataset change
- The dataset versions and file hashes used for each training or fine tuning run.
- The base model revision and the chat template used.
- Who approved the run and when.
- The evaluation results compared with the previous version.
These records connect an odd behavior back to a data source, which is the difficult part of a poisoning investigation. OWASP recommends tracking data origins and transformations, and these logs are how that tracking stays useful in practice.
Raw text: log carefully
Full prompts and outputs are the most useful and the most sensitive. They can contain personal data, credentials pasted by users and proprietary material. Options:
- Log identifiers by default, and keep full text for a sample or for flagged requests.
- Redact known patterns before storage.
- Apply a short retention period and restrict access.
- Decide in advance who may read them and under what approval.
Compare sampling and redaction choices in prompt logs for debugging.
Protect the logs
Logs are an attractive target. Store them with the same access control as the data they reflect, and make them write once so an incident cannot be tidied away afterward. Time sync across services matters, or your timeline will not line up.
Using the logs in a drill
Once a quarter, pick a past request and ask a teammate to reconstruct it from logs alone: template version, model, documents, tool calls. Gaps found in a drill cost nothing. The same gaps found mid incident cost hours.
Retention
Keep identifiers and hashes for as long as the model or dataset is in use, plus a period your compliance team sets. Keep raw text as long as you need it and no longer, since each day adds exposure.
Quick check
Do I need to log the model version if I use an API alias?
Yes. Aliases can point to different models over time. Log the exact version the response reports where it is available, and pin versions for production calls.
Is logging document IDs enough for a retrieval incident?
It tells you which documents were involved. Pair it with an index snapshot so you can see what those documents said at the time.
Sources
Last reviewed: 19 October 2026.
