An agent that can only write text is a nuisance when it is wrong. An agent whose tools can start a conveyor, move a robot arm or change a controller setting is a safety problem when it is wrong. The model has not changed. The tools have.
On 3 December 2025, CISA and eight partner agencies published guidance on integrating AI into operational technology. One sentence in Principle 4 frames this whole post: "Ultimately, humans are responsible for functional safety." This post turns that sentence into design rules for the agent's tools and permissions.
Safeguard does not move vehicles, robots or factory lines. What it can contribute is control over what an agent's tools are allowed to do, using MCP tool gating and MCP Guardrail, plus audit logs. The same design rules apply whatever platform sits between the agent and the machine.
What's affected
Anything where a language model chooses actions and a tool carries them out: a maintenance assistant that can adjust setpoints, a fleet agent that can send a command to a robot, a coding agent with access to a deployment pipeline that reaches an industrial network.
OWASP names the underlying problem as Excessive Agency, the vulnerability that "enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction." It lists three root causes: excessive functionality, excessive permissions and excessive autonomy. Each one has a physical version.
- Excessive functionality. The agent was given a tool that "includes the ability to modify and delete" when it only needed to read. In a plant, a tool that reads a sensor and also writes a setpoint.
- Excessive permissions. The tool connects "using an identity that not only has SELECT permissions, but also UPDATE, INSERT and DELETE." In a plant, a service account with write access to every controller.
- Excessive autonomy. An action with real impact runs "without any confirmation from the user."
How to check your design
Write down every tool the agent can call, then classify each into one of four tiers.
| Tier | What the tool does | Rule |
|---|---|---|
| Read | Reads state, logs, documents | Allowed by default, scoped per asset |
| Propose | Drafts a change, a work order or a command | Allowed, output goes to a person |
| Act with approval | Changes equipment state | Blocked until a named human approves this exact action |
| Never | Overrides interlocks, disables safety functions, edits safety logic | No tool exists |
Then test two things. First, can the agent reach a tool in the "Never" row by any route, including chaining two allowed tools? Second, if the agent is manipulated by text in a document or a ticket (prompt injection), what is the worst action available to it? That second answer is your real blast radius.
What to do now
- Start with read only. The CISA guidance says to "limit active control of OT infrastructure by AI without a human in the loop to account for safety concerns and latency limitations." An agent that can read and propose already saves most of the time.
- Give the agent its own identity. Do not reuse an operator's account. The guidance says to configure logging "so AI decisions can be tracked for compliance and forensic analysis, and so the logged AI identity is distinct from any typical machine or user identifiers."
- Gate tools per tenant and per tool. In Safeguard's MCP server every tool has its own feature flag. A new workspace starts with six read only tools, every other tool ships disabled until an administrator turns it on, and if tool resolution cannot complete the server falls back to the small default set rather than exposing more. That fail closed shape is the right default for any tool layer near equipment.
- Put an approval step on every state change. The approval should name the asset, the action and the value, not a generic "OK".
- Make the stop independent of the agent. The agent must not be able to delay, filter or reinterpret a stop. The Machinery Regulation, Regulation (EU) 2023/1230, Annex III, section 1.2.1, says for control systems that "the machinery or related product shall not be prevented from stopping if the stop command has already been given" and "shall not start unexpectedly."
- Bound the action space. The same section, for systems with self evolving behaviour, says they "shall not cause the machinery or related product to perform actions beyond its defined task and movement space." Enforce limits in the control layer, not in the prompt.
- Log inputs and outputs. CISA's Principle 4.1 says to "log and monitor inputs and outputs" for AI components and to keep a known good state so you know when to restore from backup.
A sketch of the approval wrapper, to show the shape. This is illustrative Python, not Safeguard syntax.
READ_TOOLS = {"get_sensor", "list_alarms", "read_manual"}
APPROVAL_TOOLS = {"set_speed", "start_cycle"}
def call_tool(agent_id, name, args, approval=None):
audit(agent_id, name, args, approval) # log first, always
if name in READ_TOOLS:
return run(name, args)
if name in APPROVAL_TOOLS:
if not approval or not approval.matches(name, args):
return {"error": "approval required", "action": name, "args": args}
return run(name, args)
return {"error": "tool not available"} # default deny
# The emergency stop never goes through this function.
Why it matters
The EU AI Act points the same way. Article 14 on human oversight says high risk AI systems must be designed so persons can "intervene in the operation of the high-risk AI system or interrupt the system through a 'stop' button or a similar procedure that allows the system to come to a halt in a safe state." Article 12 requires that high risk systems "technically allow for the automatic recording of events (logs) over the lifetime of the system." Whether your system is classified high risk or not, those are good design requirements for an agent near a machine.
The CISA guidance also warns that with active AI systems directly influencing control, "problems can escalate before operators become aware of them." Approval steps buy the time for an operator to notice.
Do not read this as "keep AI away from machines." Read it as: decide which tier each action is in, and make the system unable to move up a tier on its own.
Try it yourself
List the tools your agent can call. Put each in one of the four tiers. Then check whether any "Never" action can be reached by chaining two others. The guardrail demo shows how limits on an agent's tool calls look in practice.
Click through it at your own pace, or open it full screen.
Quick check
Why should the emergency stop not pass through the agent's tool layer?
Because a stop that depends on the agent can be delayed or misread by it. Annex III, 1.2.1 of the Machinery Regulation says machinery must not be prevented from stopping once a stop command has been given.
What are the three root causes of Excessive Agency?
Excessive functionality, excessive permissions and excessive autonomy.
Why give the agent a separate identity in logs?
So you can tell later whether an action came from a person, a machine process or the agent. CISA's guidance says the logged AI identity should be distinct from other machine or user identifiers.
Sources
- CISA and partners, Principles for the Secure Integration of Artificial Intelligence in Operational Technology (3 December 2025)
- OWASP GenAI, LLM06:2025 Excessive Agency
- Regulation (EU) 2023/1230, Machinery Regulation, Annex III
- Regulation (EU) 2024/1689, AI Act, Articles 12 and 14
- Safeguard docs, MCP tool availability and gating
