LLM05 improper output handling: a checklist for engineering leads

Improper output handling is a management problem as much as a coding one: teams ship features that pass model text to browsers, databases and tools without anyone owning the check. Leads should ask for a sink inventory, require encoding and parameterization, limit model privileges, and demand evidence in CI.

Why does this land on a lead's desk?

OWASP describes LLM05 as insufficient validation, sanitization and handling of model outputs before they pass to other components. In practice each feature team makes its own choice. One team renders Markdown safely, another pastes model text into a SQL string because the demo worked.

The OWASP page notes that the damage depends on context: excessive LLM privileges, indirect prompt injection, third-party extensions that do not validate input, missing output encoding, weak logging, and no rate limiting or anomaly detection all raise the impact. Most of those are decisions someone with authority can change.

What should you ask each team?

Bring five questions to a design or release review.

1. Where does model output go after it leaves the model call? Ask for names of components, not "the frontend".
2. Is any model text ever executed, evaluated, or used to build a command, query, path or URL?
3. What privileges does the service that holds the model call have? Could it delete data or send email?
4. What encoding or validation sits between the model and each consumer?
5. How would we notice if the model started producing unusual output?

If a team cannot answer the first question in a sentence, there is no inventory yet. That is the first deliverable.

What controls should you require?

The OWASP page gives a short list. Turn it into standards your teams can be held to:

• Treat the model like any other user. Its responses get the same validation as untrusted input.
• Follow ASVS style input validation and sanitization on every consumer.
• Encode output for its context: HTML encoding for web content, SQL parameterization for queries.
• Use prepared statements for every database operation that involves model output.
• Apply a strict Content Security Policy where model output is displayed.
• Log and monitor output patterns.

Add two of your own that follow from the risk description. Cap what the model's tools can do, and ban eval and shell execution of model text outright, with an exception process.

What evidence shows it is done?

Ask for artifacts, not assurances.

• A data flow diagram per model-backed feature with sinks marked.
• A test in CI that sends marker strings through each sink and asserts they arrive encoded or bound. Marker strings are inert, so no attack content needs to live in the repository.
• A code search result showing no exec, eval or shell calls fed by model text.
• A CSP header check on pages that render output.
• A dashboard or saved query for unusual output patterns, with a named owner.

Review these at release gates the same way you review dependency scan results.

Where do teams usually slip?

Three patterns come up in reviews. First, output is safe in the main UI but unsafe in a secondary surface such as an email notification, an admin console or a log viewer. Second, an agent tool accepts a free-form string and passes it to a shell, which is the OWASP exec example in disguise. Third, teams rely on a system prompt telling the model not to emit risky content. A prompt is a request, not a control.

What to do

1. Require a sink inventory for every feature that calls a model, within one sprint.
2. Make encoding, parameterization and CSP part of the definition of done.
3. Block eval and shell execution of model text in code review rules.
4. Add marker-string tests to CI for each sink.
5. Reduce the privileges of the service account behind the model call.
6. Assign an owner for output monitoring and review it monthly.

Quick check

Sources

• OWASP GenAI: LLM05:2025 Improper Output Handling

Last reviewed: 18 October 2026.