Monitor Before You Block: Safeguard AI Gateway and OWASP LLM Top 10

Safeguard's AI Gateway (LLM firewall, early access) sits between your application and the model provider and checks every prompt, response and tool call for prompt injection, jailbreak attempts, personal data leaving in output and output format violations. Today it runs in monitor mode: it records a guardrail event for each decision and lets traffic through unchanged. Enforce mode, which redacts or blocks, is gated behind a feature flag and an explicit per tenant opt-in, and inline enforcement on live traffic is still rolling out.

That makes the right first move simple. Route one application through the gateway in monitor mode, read what it would have blocked, fix the noisy rules, and only then decide which checks deserve to block. This post explains why, maps the gateway against the OWASP Top 10 for LLM Applications 2025, and is plain about what a gateway can never catch.

Key takeaways

• AI Gateway / LLM firewall: early access. Monitor mode is available today. Enforce mode needs the ff.aispm_gateway feature flag plus a per tenant policy, and Safeguard's docs say inline enforcement is rolling out.
• Default behaviour is fail-open with an alert. If the gateway errors, times out or goes over its evaluation budget (150 ms by default, per the docs), traffic passes and an alert is raised. Fail-closed is opt-in.
• Gateway decisions are guardrail events, not findings. Each event carries direction, verdict, category, model name, rule, mode and latency overhead. Raw prompt and response text is never stored, according to the docs.
• It covers parts of four OWASP LLM risks well (prompt injection, sensitive information disclosure, improper output handling, unbounded consumption) and touches a few more. It does nothing for data and model poisoning in training, and little for vector store access control.
• Related capabilities and their labels: AI-BOM discovery (early access) finds the LLM call sites to protect; AI-SPM model artifact scanning (early access) checks the weights you load; MCP Guardrail (early access) governs agent tool traffic to MCP servers; Guard runtime protection (early access) and the Package Firewall (early access) follow the same monitor then enforce pattern; policies, gates and guardrails (GA) and agent identity (GA) cover the build side and the access side.
• The gateway does not replace output validation in your own code. OWASP's guidance on improper output handling still applies to you.

The problem: model traffic is an unwatched channel

Most teams that ship an LLM feature log their HTTP APIs and scan dependencies in CI, but the calls to a model provider get none of that. A prompt goes out, a response comes back, and the application acts on it. Increasingly the response picks a tool, fills in arguments, writes a query, or drafts a message that will be sent.

Three properties make this channel different from ordinary API traffic.

First, the instructions and the data share one stream. A model cannot reliably tell the system prompt you wrote from a sentence hidden in a web page your retrieval step pulled in. OWASP describes indirect prompt injection as exactly this: the model accepts input from external sources such as websites or files, and content in them alters its behaviour.

Second, the output is generated, not selected. The same input can produce different responses, and a response can contain personal data, internal configuration or markup that your front end will happily execute if nobody encodes it.

Third, the failure is quiet. A successful injection does not throw an exception. The application keeps working, just not for you.

The standards bodies have noticed. NIST's Generative AI Profile, the EU AI Act, the UK NCSC and CISA joint guidelines and MITRE ATLAS all ask for some combination of input monitoring, output checking, logging and a way to stop a misbehaving system. A gateway is one practical way to get the first three in place quickly. It is not the only control you need, and the rest of this post is about where the line sits.

Which OWASP LLM Top 10 2025 risks a gateway addresses, and which it cannot

The OWASP Top 10 for LLM Applications 2025 lists ten risks, LLM01 through LLM10. A runtime gateway can only see what passes through it at inference time: prompts, retrieved and tool supplied content that is folded into prompts, responses and tool calls. Anything that happens before deployment, in training data, in model weights or inside your vector store's access rules, is out of its sight.

The table below maps each risk to the checks the Safeguard AI Gateway (early access) documents, and to what you still need. "Partial" means the gateway gives you a useful signal or control, not a guarantee.

LLM01 Prompt Injection: the gateway's main job, with a caveat

OWASP is unusually direct about this risk. Its LLM01 page says that, given how models work, it is unclear whether fool-proof prevention for prompt injection exists. Its mitigation list is mostly architectural: constrain model behaviour, define and validate output formats, filter input and output, enforce least privilege, require human approval for high risk actions, segregate external content, and test adversarially.

A gateway delivers the "input and output filtering" item and helps with "validate output formats". Safeguard's docs describe three prompt side checks: direct injection with pattern and classifier detection, indirect injection that scans retrieved or tool supplied content rather than only the user's turn, and jailbreak patterns. The indirect check matters most for retrieval augmented generation and agent workflows, because that is where untrusted text enters without a human typing it.

The caveat is the same one OWASP gives. Pattern and classifier detection is probabilistic. OWASP's own scenarios include multilingual and encoded instructions written to slip past filters. Treat the gateway as a detector that raises the cost of an attack and gives you visibility, not as the thing that makes injection impossible. The least privilege and human approval items still have to live in your application.

LLM02 Sensitive Information Disclosure: redaction on both directions

OWASP's LLM02 covers personal data, financial and health details, confidential business data and more, leaking through model output. It also notes that restrictions written into the system prompt "may not always be honored", because prompt injection can bypass them.

That is the argument for an external check. The AI Gateway's docs list personal data in the prompt (redact) and personal data egress in the response (redact). The sample policy redacts patterns such as email addresses and national identifier formats and replaces them with a fixed marker. Redacting on the way in keeps personal data from reaching the provider at all; redacting on the way out stops a response from carrying it to the user or a downstream system.

What the gateway cannot do is decide who should see what. OWASP's access control items, least privilege on data and restricted data sources, sit in your retrieval layer and your identity system.

LLM05 Improper Output Handling: schema checks help, encoding is still yours

OWASP defines this risk as insufficient validation, sanitisation and handling of model output before it is passed downstream. The examples are familiar web flaws with a new source: model output passed to a shell, rendered as script in a browser, or concatenated into SQL.

The gateway's output format and schema enforcement, documented as "block on violation", is useful when your application expects structured output. If the response is supposed to be a JSON object with three fields and it is not, blocking it is better than parsing whatever arrived. But a response can be perfectly valid JSON and still contain a string that becomes cross-site scripting when your front end renders it without encoding. OWASP's mitigations here, treating the model like any other user, encoding output for its context, using parameterised queries and a strict content security policy, are application code. No gateway can do context aware encoding for a template it never sees.

LLM06 Excessive Agency: the gateway sees calls, you set permissions

OWASP traces excessive agency to three roots: excessive functionality, excessive permissions and excessive autonomy. Its mitigations are to minimise extensions and their functionality, avoid open ended extensions like "run a shell command", minimise permissions, execute in the user's context and require user approval for high impact actions.

The AI Gateway evaluates tool calls in the traffic it sees, which gives you a record of what the model tried to do. That record is valuable evidence when you trim tool lists. But the gateway cannot shrink the permission set of the database account your tool uses. For agents that call MCP servers, Safeguard's MCP Guardrail (early access) generates guardrail proxy URLs that govern agent tool traffic, and agent identity (GA) tracks which agent holds what access at what level. Both are covered in more depth in our MCP Guardrail post.

LLM07, LLM09 and LLM10: useful signals, not fixes

For system prompt leakage, OWASP makes a point worth repeating: the system prompt should not be treated as a secret or used as a security control. Its own mitigations include implementing guardrails outside the LLM, because an independent system that inspects output is preferable to instructions inside the prompt. A gateway is that independent system for output checks. The deeper fix, keeping credentials and authorisation decisions out of prompts, is design work.

For misinformation, the gateway's docs list hallucination detection with a default verdict of monitor. That is the right default. A probabilistic hallucination signal is good for dashboards and sampling for human review; it is not good enough to block answers automatically in most products.

For unbounded consumption, OWASP recommends input validation on size, rate limiting and user quotas, timeouts and throttling, and monitoring. The gateway applies per tenant rate and abuse limits (the sample policy uses requests per minute and a burst allowance) and bounded inspection. Per user quotas and spend alerts still belong in your application and your provider account.

LLM03, LLM04 and LLM08: outside a gateway's view

Supply chain, poisoning and vector store weaknesses mostly happen before or beside the inference call, where a gateway cannot look. For model files, Safeguard's AI-SPM (early access) scans model artifacts such as pickle, PyTorch, safetensors, GGUF and ONNX files. For inventory, AI-BOM discovery (early access) finds SDK imports, model files, prompt templates and MCP server configuration in code. The AI-BOM discovery post in this series walks through it.

What the EU AI Act, NIST AI 600-1 and GDPR imply for runtime controls

None of these texts says "deploy an LLM firewall". Each asks for outcomes that a monitored request path makes easier to show.

EU AI Act Article 15: accuracy, robustness and cybersecurity

Article 15 of Regulation (EU) 2024/1689 applies to high-risk AI systems. Most chat assistants and coding helpers are not high-risk under the Act; systems used in the areas the Act lists as high-risk can be. If yours is, the article's text, as published on the Commission's AI Act Service Desk, contains three points that matter for runtime controls:

• Article 15(1) requires high-risk systems to achieve an appropriate level of accuracy, robustness and cybersecurity and to perform consistently in those respects throughout their lifecycle. "Throughout their lifecycle" implies watching the system after deployment, not only testing it before.
• Article 15(4) requires systems to be as resilient as possible to errors, faults or inconsistencies, and says robustness may be achieved through technical redundancy, which may include backup or fail-safe plans. That is a direct prompt to decide, in writing, what happens when a control like a gateway fails.
• Article 15(5) requires resilience against unauthorised third parties trying to alter use, outputs or performance by exploiting vulnerabilities. It names measures to prevent, detect, respond to, resolve and control for data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks and model flaws.

A gateway speaks to the "detect" and "respond" parts of 15(5) for inputs designed to make the model misbehave and for confidentiality attacks at inference time. It does not address data or model poisoning. Check the Service Desk timeline for when the high-risk obligations apply to your system; this post does not restate dates.

NIST AI 600-1: the Generative AI Profile

NIST AI 600-1, published in July 2024, is a profile of the AI Risk Management Framework for generative AI. It lists twelve risks unique to or worsened by generative AI, including Confabulation, Data Privacy, Information Integrity, Information Security and Value Chain and Component Integration.

Its Information Security section describes direct prompt injection and indirect prompt injection, where adversaries inject prompts into data likely to be retrieved, in terms that match OWASP's. Several suggested actions map to runtime monitoring:

• MG-4.1-002 asks organisations to establish and evaluate processes for post-deployment monitoring of generative AI systems, particularly for confabulation and cyber risks.
• MS-2.7-004 suggests identifying metrics for the effectiveness of security measures, such as the number of unauthorised access attempts, bypass and extraction. Guardrail event counts by rule are exactly that kind of metric.
• MS-2.7-007 suggests red-teaming to assess resilience against prompt injection and other attacks.
• MANAGE 2.4 asks for mechanisms to supersede, disengage or deactivate AI systems whose outcomes are inconsistent with intended use, and MG-2.4-004 asks for criteria that warrant deactivation.

The MANAGE 2.4 point is the quiet argument for having an enforce mode available even if you rarely use it. Monitoring tells you something is wrong; a block or throttle path lets you act without redeploying.

GDPR Article 32: security of processing

If prompts or responses can contain personal data, GDPR Article 32 applies to that processing. The EDPB's guide for small businesses summarises the duty: controllers and processors must put in place appropriate technical and organisational measures to secure personal data at a level appropriate to the risk, taking into account the state of the art. The same guide names pseudonymisation as one of the measures the GDPR recommends.

Two practical consequences follow. Redacting personal data before it reaches a model provider reduces what that provider processes. And your own monitoring must not become a new store of personal data. Safeguard's docs say gateway events carry redacted metadata only and raw prompt and response text is never persisted, which helps with the second point. You still need to account for the gateway in your records of processing and any data protection impact assessment, because the gateway itself processes the traffic in flight.

NCSC and CISA guidelines, and MITRE ATLAS

The UK NCSC's Guidelines for secure AI system development, developed with CISA and international partners, split the lifecycle into secure design, development, deployment, and operation and maintenance. The operation section asks you to monitor your system's behaviour, so you can see sudden and gradual changes that affect security, and to monitor and log your system's input, such as prompts, in line with privacy requirements, including detection of adversarial inputs.

MITRE ATLAS gives you shared names for what you are watching for. In the September 2026 ATLAS data, AML.T0051 is LLM Prompt Injection, with sub-techniques for Direct (AML.T0051.000), Indirect (AML.T0051.001) and Triggered; AML.T0054 is LLM Jailbreak; AML.T0057 is LLM Data Leakage. Two mitigations line up with a gateway: AML.M0020 Generative AI Guardrails, described as controls placed between users, tools and models that evaluate prompts, retrieved context, outputs and agent actions, and AML.M0024 AI Telemetry Logging, logging inputs and outputs of deployed models. Tagging your gateway rules with these IDs makes reporting to a security team or auditor much easier.

Anthropic's own documentation on mitigating jailbreaks and prompt injections draws the same line between threat models: jailbreaks and direct injection, where the user is the adversary, and indirect injection, where the user is trusted but the model reads hostile third-party content. It recommends input screening, prompt design, and delivering untrusted content in tool results with a clear statement that it is data, not instructions. A gateway complements those model-side practices; it does not replace them.

How Safeguard handles it: the AI Gateway in the request path

Safeguard's docs describe the AI Gateway as a runtime enforcement point for the AI in your applications, "the analog of the Package Firewall" for prompts, responses and tool calls instead of dependencies. It evaluates every LLM interaction against a set of guardrails and can observe, redact or block.

Inline proxy or SDK shim

The docs give two ways to put the gateway in the path: as an inline proxy between your app and the model provider, or as a thin SDK shim around your LLM client.

With an inline proxy, your application sends model requests to the gateway, which evaluates them, forwards them to the upstream provider named in your tenant's policy, and evaluates the response on the way back. Prompt logic does not change; the cost is a network hop and making sure every caller uses the proxy. An SDK shim wraps the client library inside your process instead, avoiding the extra hop but requiring the shim in each application.

The docs do not publish a client-side setup snippet or a CLI command for either path. They state plainly that there is no CLI command for running the gateway yet. Because the gateway is early access and configured per tenant, setup runs through Safeguard rather than a self-serve screen we can show you. We describe outcomes below rather than invent commands.

Where it fits among Safeguard's enforcement points

Safeguard's guardrails documentation lists six enforcement points: IDE, commit, CI, registry, admission and runtime. The runtime row includes LLM calls and MCP tool calls checked against your guardrails, marked early access and delivered by Safeguard Guard at the library and framework level inside your application. The AI Gateway is the dedicated LLM traffic version of that runtime idea. The build side is handled by policies, gates and guardrails (GA), which the platform evaluates on the scans it runs for connected repositories. The gateway is a separate runtime mechanism with its own configuration.

Monitor versus enforce: what each does today

The docs define two modes.

Monitor. Every prompt, response and tool call is evaluated, a guardrail event is recorded, and traffic passes through unchanged. The docs call this "the safe way to baseline what enforcement would do before turning it on". Monitor is the default everywhere, and it is the mode available today.

Enforce. The gateway acts on its verdict: it redacts personal data in either direction, or blocks injection and jailbreak attempts, according to your policy. Enforce mode requires the ff.aispm_gateway feature flag and an explicit per tenant opt-in. The docs' rollout note says the decision core, guardrail checks, monitor telemetry path and safe-fail behaviour are built, while inline proxy enforcement on live traffic is rolling out.

Why make monitor the default? Because the cost of a false positive in an LLM product is high and visible. A block on a legitimate customer question looks like an outage. A redaction that strips an order number out of a support answer breaks the answer. Classifier thresholds tuned on someone else's traffic will not match yours. Running in monitor first gives you evidence about what each rule would have done, on your real prompts, before any user feels it. That evidence is also what keeps a control switched on after its first awkward incident ticket.

The guardrail checks, direction by direction

The docs list each check with the direction it applies to and its default verdict. We reproduce it here in our own words.

The indirect injection row deserves emphasis. The docs say retrieved content, meaning RAG results and tool output, is untrusted, so the gateway inspects it as well as the human's input. That matches OWASP's LLM01 and NIST AI 600-1's description of indirect injection through data likely to be retrieved.

Step by step: route one application through the gateway in monitor mode

This walkthrough uses only what Safeguard's docs state. Where the docs do not show a screen or command, we describe the outcome you should expect and leave the mechanics to your Safeguard contact, because the AI Gateway is early access and configured per tenant.

1. Pick one application and one environment. Choose an LLM feature with real but bounded traffic, ideally a staging or internal deployment that sees realistic prompts. Avoid starting with your highest volume production path. You want enough events to learn from and a low cost if something goes wrong.

2. Find every place that application calls a model. If you use AI-BOM discovery (early access), an administrator enables it through the ff.aispm feature flag, and discovery runs across connected repositories, cloud accounts and SaaS integrations through the standard scan flow. In code, it finds imports of common AI SDKs, model weight files, prompt templates, notebooks and MCP server configuration. The outcome is a list of call sites. Any call site you miss is traffic the gateway will never see.

3. Ask Safeguard to enable the AI Gateway for your tenant. The gateway is early access and configured per tenant. The docs say there is no CLI command for running it yet, so this step is a conversation, not a command. Confirm which insertion method, inline proxy or SDK shim, fits your architecture.

4. Set a monitor mode policy. The docs show a representative per tenant policy. This is an excerpt, with each line exactly as the docs show it; the full sample on the docs page also includes rate and redaction settings.

   aigateway:
     mode: monitor                 # monitor | enforce
     fail_mode: open               # open (never break the app) | closed
     eval_budget_ms: 150           # per-request eval cap; over budget -> pass through + alert
     provider: { name: anthropic, upstream: "https://api.anthropic.com" }
     checks:
       injection: block
       indirect_injection: block
       pii_egress: redact
       jailbreak: block
       toxicity: monitor
       hallucination: monitor
   

   Read it carefully. mode: monitor means nothing is blocked or redacted on live traffic, even though individual checks say block or redact. Those per check values describe what enforce mode would do, and the guardrail event records that verdict. The docs describe the default policy the same way: monitor mode, fail-open, injection and jailbreak set to block, personal data egress set to redact.

5. Put the gateway in the path. With an inline proxy, your application's model client sends requests to the gateway, which forwards them to the upstream in your policy. With an SDK shim, the wrapper inside your process does the evaluation. The docs do not publish the client-side configuration, so we will not invent one. The outcome to verify is simple: every request from the call sites in step 2 produces a guardrail event.

6. Send representative traffic. Let normal users or internal testers use the feature. Make sure the traffic includes retrieval results and tool outputs if your application uses them, because the indirect injection check only has something to inspect when that content flows. Add your own red team prompt set if you have one; NIST AI 600-1 MS-2.7-007 recommends red teaming for prompt injection. This post does not include attack strings.

7. Read the guardrail events. Each event carries the direction (prompt or response), the verdict, the category, the model name, the rule that fired, the mode, and the latency overhead the gateway added. Events are tenant scoped and audited. Raw prompt and response text is not stored, so you will see redacted metadata, not the conversation.

8. Triage what monitor mode found. Group events by rule and verdict. For each rule that would have blocked, sample cases with the application team and decide whether they were real attacks, harmless edge cases or a pattern your product legitimately uses. Check the latency overhead against the 150 ms budget, look for over budget pass-through alerts, and look for events flagged as truncated, which the docs say happens when a body exceeds the configured inspection limit.

9. Decide, rule by rule, what to enforce. Some checks are good candidates for enforcement early: personal data redaction on responses, and schema enforcement where your application already expects strict structure. Others, such as hallucination, should stay in monitor. Enforcement needs the ff.aispm_gateway flag and an explicit per tenant opt-in, and inline enforcement is rolling out, so confirm with Safeguard that it is available for your tenant before you plan a date.

10. Keep your application-side controls. Whatever you enforce, keep output encoding, parameterised queries, least privilege for tools and human approval for high impact actions in your code. The gateway is a second line, not the first.

Reading guardrail events: what good looks like

Look for three patterns. If a few rules produce most would-block events, that is where to tune. If indirect injection events cluster around one document collection or tool, fix the content at its source. If latency overhead approaches the budget or pass-through alerts are frequent, some traffic is going unchecked, and you need to know that before you rely on enforcement.

One important difference from the rest of Safeguard: the docs say gateway decisions are high volume runtime telemetry, so they are recorded as guardrail events rather than as standard findings. Static AI issues, such as model scans and agent or MCP analysis, still flow into the unified findings model as AI security findings, which the unified findings docs show under the "AI Security" scanner filter. Do not expect gateway events to appear in the same findings table as your SCA results.

Fail-open versus fail-closed

Safeguard's docs are explicit about safe-fail design:

• Fail-open by default. On any gateway error or timeout, traffic is allowed through and an alert is raised. Fail-closed is available only if a tenant explicitly opts in.
• Resource capped. Each request has an evaluation budget, 150 ms by default. Over budget or under backpressure, the gateway passes traffic through with an alert rather than blocking the application.
• Bounded inspection. Bodies larger than the configured limit are truncated for scanning and flagged as truncated.

The reasoning is availability. A security control in the request path that can take the product down will be removed. Fail-open means the worst case of a gateway fault is the situation you had before the gateway existed, plus an alert.

The cost is real, and you should say it out loud in your risk register. During a fault, an overload or a long body, some traffic passes unchecked. An attacker who can make the gateway slow, or who can push malicious content past the inspection limit, gets through in those moments. The alerts give you visibility, not prevention.

When would you choose fail-closed? Consider it where an unchecked response is worse than no response:

• The model's output directly triggers an action that moves money, changes access or deletes data, and there is no human approval step.
• The feature serves a regulated decision where Article 15(4) style fail-safe planning applies and your risk assessment says "stop" is the safe state.
• The traffic is low volume and internal, so a short outage costs little.

There is also a middle path that often works better than flipping the whole gateway: keep the gateway fail-open, and make the high impact actions in your own code fail-closed. A tool that issues refunds can require a human approval when no gateway verdict is present, regardless of what the gateway does. That follows OWASP LLM06's advice to require user approval for high impact actions, and it keeps the rest of the product available. Safeguard's desktop and mobile apps (early access, beta) apply the same idea to commands run by the Safeguard CLI, as covered in our post on human approval for agent actions.

Moving from monitor to enforce: a decision table

How the AI Gateway relates to MCP Guardrail, Guard and the Package Firewall

Safeguard has several controls that sound alike. They sit at different points.

The docs make the Package Firewall comparison themselves: it is "the same monitor/enforce pattern for dependencies". The analogy is useful. In both cases you put a checkpoint on a fetch path, watch decisions before enforcing them, and accept that anything which bypasses the path is not checked.

The difference between the AI Gateway and MCP Guardrail is the counterparty. The gateway watches the conversation with the model. MCP Guardrail watches the conversation between an agent and the tools it calls. An agent can receive a clean response from the model and still call a dangerous tool with dangerous arguments. If your application uses MCP, you want both views. The how MCP Guardrail works post and the Guard monitor then enforce post go further on each. For the wider picture of AI-SPM, the gateway and AI-BOM together, see AI-SPM, gateway and AI-BOM.

What a gateway will never catch, and why model and prompt design still matter

A gateway inspects text at a boundary, so any gateway has fixed blind spots. It will miss some instructions that read like ordinary prose; OWASP notes injections need not even be human-readable. It cannot see harm that only appears after your code acts, such as a safe looking string concatenated into a query. It cannot see traffic that bypasses it, such as a batch job calling the provider directly. It cannot see poisoning that happened in training, or an authorisation bug that lets your retrieval layer hand the model documents the user should never see.

This is why the model-side and design-side practices still matter. OWASP LLM07 recommends keeping sensitive data and authorisation logic out of system prompts and enforcing critical controls outside the model in a deterministic, auditable way. Anthropic's documentation recommends delivering untrusted third-party content inside tool results and stating in the system prompt that such content is data, not instructions. OWASP LLM06 recommends minimal tools and permissions. None of these is replaced by a gateway; each makes the gateway's job smaller.

Requirement versus what Safeguard does versus what you still own

What Safeguard does not do

• It does not offer the AI Gateway as a generally available product. The AI Gateway is early access. Monitor mode is available; enforce mode requires a feature flag and per tenant opt-in, and the docs say inline enforcement on live traffic is rolling out. Do not plan a compliance commitment around enforcement until Safeguard confirms it for your tenant.
• It does not give you a self-serve setup path or CLI for the gateway. The docs state there is no CLI command for running the gateway yet. Setup is per tenant, with Safeguard.
• It does not guarantee detection. Injection, jailbreak, toxicity and hallucination checks are pattern and classifier based, which means misses and false positives. OWASP itself says it is unclear whether fool-proof prevention of prompt injection exists.
• It does not check traffic during faults when fail-open is set. By design, on error, timeout, over budget or backpressure, traffic passes with an alert. Oversized bodies are only partly inspected.
• It does not replace output validation in your application. Encoding for HTML, SQL or shell contexts, parameterised queries and content security policies remain your code.
• It does not set permissions for your tools or data. Least privilege, user context execution and approval steps for high impact actions are design decisions in your systems.
• It does not address training data or model poisoning. That needs dataset provenance and model evaluation. AI-SPM (early access) scans model artifacts, which is a different control.
• It does not store conversations for later replay. Raw prompts and responses are not persisted, which is good for privacy and means you cannot reconstruct the full text of an incident from gateway events alone. Plan your own logging if you need that, with a privacy review.
• It does not make you compliant with the EU AI Act, GDPR or any framework. It provides controls and evidence that can support your own assessment.

Common mistakes

1. Starting in enforce mode. Turning on blocking before you have a baseline is how a guardrail gets turned off after its first false positive. Start in monitor, which is also the only mode generally available today.
2. Reading the per check block values as live behaviour. In monitor mode, a check set to block records a block verdict but does not block. Make sure your stakeholders understand the difference before they read the event counts.
3. Protecting one call site and forgetting the rest. Background jobs, admin tools and evaluation scripts often call the provider directly. Use an inventory, such as AI-BOM discovery (early access), and verify that every call site produces events.
4. Treating the system prompt as a security boundary. OWASP LLM07 is clear that it is not. Move credentials and authorisation checks out of prompts.
5. Ignoring pass-through alerts. Fail-open alerts are the gateway telling you it did not check something. If they are frequent, your coverage is lower than your dashboard suggests.
6. Enforcing hallucination detection. A probabilistic quality signal makes a poor blocking rule. Use it for sampling and review.
7. Skipping the privacy review because "the gateway does not store prompts". The gateway still processes personal data in flight. It belongs in your records of processing.

FAQ

What is an LLM firewall?

An LLM firewall, or AI gateway, is a checkpoint between an application and a model provider that inspects prompts, responses and tool calls against rules. MITRE ATLAS describes the general idea as generative AI guardrails: controls placed between users, tools and models that evaluate inputs, retrieved context, outputs and agent actions. Safeguard's AI Gateway is one such checkpoint and is early access.

Is Safeguard's AI Gateway generally available?

No. It is early access. Monitor mode is available today. Enforce mode is gated behind the ff.aispm_gateway feature flag and per tenant opt-in, and Safeguard's docs say inline enforcement is rolling out.

Does an AI gateway stop prompt injection?

It reduces risk and gives you visibility; it does not stop all prompt injection. OWASP says it is unclear whether fool-proof prevention exists. Combine the gateway with least privilege for tools, human approval for high impact actions and separation of untrusted content.

What is the difference between monitor mode and enforce mode?

In monitor mode, every interaction is evaluated, a guardrail event is recorded and traffic passes unchanged. In enforce mode, the gateway acts on its verdict by redacting personal data or blocking injection and jailbreak attempts, according to your policy.

Why does the AI Gateway fail open by default?

So that a gateway fault never breaks your application. On error, timeout or when the evaluation budget is exceeded, traffic passes and an alert is raised. Fail-closed is available on explicit opt-in for cases where an unchecked response is worse than no response.

Does the AI Gateway store my prompts and responses?

According to Safeguard's docs, raw prompt and response text is never persisted. Guardrail events carry redacted metadata: direction, verdict, category, model name, rule, mode and latency overhead.

Which OWASP LLM Top 10 risks can a gateway not address?

Mainly LLM03 Supply Chain, LLM04 Data and Model Poisoning and LLM08 Vector and Embedding Weaknesses, because they happen in training, in model files or in your retrieval layer rather than in inference traffic. It gives only a weak signal for LLM09 Misinformation.

Do I need an AI gateway for EU AI Act compliance?

The Act does not name gateways. Article 15 asks high-risk AI systems for appropriate accuracy, robustness and cybersecurity throughout their lifecycle, resilience to faults with possible fail-safe plans, and measures against adversarial inputs and confidentiality attacks. A gateway can support parts of that; your conformity work still covers the rest.

How is the AI Gateway different from MCP Guardrail?

The AI Gateway watches traffic between your application and a model provider. MCP Guardrail (early access) watches agent tool traffic to MCP servers through generated proxy URLs. Agent workloads usually need both.

Where to start

If you want to try the parts of Safeguard that are generally available today alongside this early access work, download the apps and CLI at safeguard.sh/download and read the documentation at docs.safeguard.sh/docs. To evaluate the AI Gateway itself, ask Safeguard about early access for your tenant and start with one application in monitor mode.

The AI Gateway watches what flows through your models at runtime; it does not fix the libraries your AI application is built on. The SDKs, frameworks and container images under an LLM feature carry ordinary vulnerabilities too, and the section below covers how Safeguard handles those.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The demos below do not show the AI Gateway, which is early access and configured per tenant. They show two generally available or beta surfaces that apply the same principle this post argues for: an AI system gets only the access you grant, and risky actions wait for a person.

This demo shows Claude connected to the Safeguard MCP server (GA) with OAuth, and how you choose which Safeguard tools Claude is allowed to use. Choosing a minimal tool set is OWASP LLM06's "minimise extensions" advice in practice, applied to your own security assistant.

Click through it at your own pace, or open it full screen.

This demo is a real desktop app session (early access, beta). The CLI session shows in the desktop app's conversation list, you ask for work on your computer, pick the folder, then Approve or Deny each command and file edit, and it ends with the upgrade done and tests passing. It is the human approval step for high impact actions that OWASP recommends, built into an agent workflow.

Click through it at your own pace, or open it full screen.

For a wider view of how these early access AI controls fit next to the GA core, see what Safeguard does today, every capability labelled and the comparison of LLM firewall and guardrail approaches for enterprise deployment.

Quick check

Sources

• Safeguard Docs: AI Gateway (LLM Firewall)
• Safeguard Docs: Guardrails and Enforcement
• Safeguard Docs: Unified Findings and Feature Flags
• Safeguard Docs: AI Security Posture Management (AI-SPM)
• Safeguard Docs: AI-BOM Discovery
• Safeguard Docs: Package Firewall
• Safeguard Docs: MCP tool gating
• OWASP Top 10 for LLM Applications 2025
• OWASP LLM01:2025 Prompt Injection
• OWASP LLM02:2025 Sensitive Information Disclosure
• OWASP LLM05:2025 Improper Output Handling
• OWASP LLM06:2025 Excessive Agency
• OWASP LLM07:2025 System Prompt Leakage
• OWASP LLM09:2025 Misinformation
• OWASP LLM10:2025 Unbounded Consumption
• NIST AI 600-1, Generative Artificial Intelligence Profile
• AI Act Service Desk: Article 15, Accuracy, robustness and cybersecurity
• EDPB: Secure personal data (SME data protection guide)
• UK NCSC: Guidelines for secure AI system development
• UK NCSC: Secure operation and maintenance
• MITRE ATLAS data, release 2026.09
• Anthropic: Mitigate jailbreaks and prompt injections

Last reviewed: 24 October 2026. Product capabilities change; check docs.safeguard.sh for the current state.