Check both sides if you can afford it. Input checks stop abusive requests before you pay for them. Output checks catch what the model produced anyway. If you can only do one, pick output for consumer products and input for cost-sensitive ones, then test it against your own abuse cases.
What each side catches
Input moderation looks at the user's message before it reaches the model. It saves money on requests you would refuse anyway and gives you a clean signal about user intent for abuse handling. It cannot see what the model will say, and it can be fooled by rephrasing.
Output moderation looks at the model's reply before the user sees it. It catches harmful text that came from an innocent-looking prompt, from retrieved documents, or from a prompt injection that never appeared in the user's message. It costs you the generation, and it adds delay.
Both sides matter because the model's input is not only the user's message. System prompts, retrieved content and tool results all feed in. OWASP's guidance on prompt injection lists filtering inputs and outputs as one mitigation among several and states that none of them fully prevents the problem.
A concrete tool and what it states
One major provider documents a free moderation endpoint that classifies text and images and returns a flagged value, per-category flags and per-category scores. Its documentation lists several limits worth knowing about before you build on any such service:
- Results can contain an error instead of scores, so your code must check the result type first and decide what to do on failure.
- With streamed responses, scores arrive only after the full output is available.
- Moderation covers tool call arguments and outputs in conversation content, but not tool names, descriptions or schemas.
- The model is updated over time, so thresholds based on scores may need recalibrating.
The same documentation says known or suspected child sexual abuse material must not be sent to the endpoint. If your product could receive that, get legal advice on reporting duties before you design the pipeline.
Fail open or fail closed
Decide in advance what happens when the filter is down or returns an error. For a children's product or a high-risk setting, fail closed and show a safe message. For a low-risk internal tool, failing open with logging may be reasonable. Write the choice down.
Where scores meet policy
A score is not a policy. Decide what you block, what you allow with a warning, and what you route to a human. Start with a conservative threshold, log blocked items without storing more personal data than you need, and review a sample weekly. Recalibrate when the model behind the filter changes.
What filters do not do
A content filter does not protect your systems from prompt injection that triggers tool actions. It reads text, and it does not stop a model from calling a tool it has access to. Limit tool permissions separately. See human approval for LLM tool calls.
Quick check
Can I rely on the model's built-in refusals instead?
They help, but they are not a control you can test or tune. Add your own checks where the harm would be serious.
Should I show users why content was blocked?
Show a short, neutral message. Detailed reasons help attackers tune their attempts.
Sources
Last reviewed: 19 October 2026.
