Agentic AI in Regulated Industries: Open Source Guardrails That Hold Up

Banks, insurers, brokers and healthcare providers can put AI agents into production, but only when every consequential action passes a policy check before it runs, carries a scoped identity, lands in a tamper evident audit log, and can be stopped by a named person. Open source standards such as the Model Context Protocol make those controls inspectable and portable.

Agents change the compliance question. A chatbot produces text that a person acts on. An agent acts: it reads a customer record, updates a claim, drafts a message, or calls another agent. Regulators have not written agent rulebooks, but their existing rules on supervision, access control, record keeping and human oversight apply directly. This guide covers two recent Linux Foundation pieces, what the EU AI Act, NIST, FINRA, the US banking agencies and HHS say as of October 2026, OWASP's agent threat list, MCP's own security rules, and a step by step build an examiner could inspect.

Key takeaways

• Linux Foundation research says the gate for enterprise agents is governance, not capability: policy enforcement before actions, identity, metering and auditability.
• The EU AI Act regulates AI systems by use, not "agents" as a category. Credit scoring and life or health insurance pricing are Annex III high risk uses, now applying from 2 December 2027 after the AI Omnibus.
• High risk systems need automatic event logs (Article 12), effective human oversight with a safe stop (Article 14), and deployers keep logs for at least six months (Article 26).
• FINRA's 2026 oversight report names agent risks: autonomy, acting beyond scope, weak auditability and sensitive data handling.
• The April 2026 interagency model risk guidance (SR 26-2) leaves generative and agentic AI out of scope but says your own governance must still cover them. That is not an exemption.
• HIPAA's Security Rule already treats "software programs" as entities that need granted access and audit controls.
• OWASP's Top 10 for Agentic Applications 2026 adds "least agency": no more autonomy than the task needs.
• MCP's specification asks for human approval of tool calls, untrusted tool annotations, audience bound tokens and no token passthrough.

Why agents change the problem for regulated firms

Regulated firms already run machine learning through model risk reviews. What is new is a model that plans several steps, chooses tools and executes them with real permissions. That changes three things.

Who acted. Systems of record show a human user ID on every change. With agents, the change may come from a shared service account, triggered by a prompt from a customer, an employee or a scraped web page. If you cannot reconstruct who asked and under whose authority, you cannot answer a supervisory question.

When control happens. Periodic review of a scoring model's outputs works. It does not work for an agent that moves data in seconds. The control has to sit in front of the action.

What the system is made of. An agent is a model, a prompt, tools served by separate MCP servers, a framework, memory and the data each tool reaches. Each part has its own version, owner and vulnerabilities. A firm that can produce an SBOM for its payments service but not for its claims agent has a gap an examiner will find.

What the Linux Foundation pieces argue

Transparency and sovereignty for finance and healthcare

Hilary Carter, SVP of Research at the Linux Foundation, wrote on 20 April 2026 about a regulated industries roundtable at the Linux Foundation's San Francisco AI Forum, held under the Chatham House Rule and facilitated by FINOS executive director Gabriele Columbro. It fed into the LF Research report Open Source and the Future of AI, co-authored with Anna Hermansen. In summary:

• Regulated firms cannot justify a denied loan or an executed trade with an opaque model. Open models, with inspectable code and weights, let institutions stress test and audit for bias.
• Sovereignty matters: a bank can tune a global foundation model on local data inside its own network and keep control of outputs.
• Shared, pre-competitive work on governance frameworks and regulatory reporting reduces everyone's risk. She cites the FINOS AI Governance Framework.
• She names Ray for compute, Goose for building agents, and MCP as the connector between agents and siloed bank data.
• Humans should be in control, defining intent and setting guardrails.

One caveat. Open weights help with inspection and hosting control, but they do not make an agent auditable on their own. Most audit evidence comes from around the model: which tools it could call, with what credentials, what it called, and who approved it. That holds whether the model is open or proprietary.

Governance as the gate for enterprise agents

Irving Wladawsky-Berger's post, on the Linux Foundation blog on 9 October 2026 (first published on his own blog on 6 October), summarises the LF Research report Enterprise Transitions to the Agentic Era by Hilary Carter, Hillary Curran and Anni Lai, based on interviews with 25 leaders and operators. The points that matter here:

• The strongest agreement was on a governance layer between agents and resources with four jobs: policy enforcement before consequential actions, identity and authorization, metering and attribution, and an immutable record of actions.
• Retrospective audits are not enough at machine speed; guardrails must act before execution.
• Accountability stays with the deploying organisation and the people who configure the agent. Accountability across chains of delegating agents is described as unresolved.
• Trust and governance ranked first among adoption obstacles.
• Shadow AI is already here, so firms should bring experiments under managed adoption.

Together the posts give a usable thesis: regulated firms can adopt agents if they build a governance layer they can inspect, and open standards make that layer portable across models and vendors.

What regulators actually require, as of October 2026

EU AI Act: obligations by use, on a later timeline

The European Commission's AI Act page lists high risk uses including AI that controls access to essential private services, with credit scoring that can deny a loan as its example. Annex III, as shown on the Commission's AI Act Service Desk, covers systems that evaluate creditworthiness or set credit scores for natural persons (fraud detection excepted) and systems for risk assessment and pricing in life and health insurance. An agent that decides or materially shapes those outcomes is a high risk system whatever its architecture. One that summarises internal documents for staff usually is not.

The timeline moved this year. According to the Commission:

The delay is not a reason to wait. The listed obligations, from risk management and logging to human oversight and cybersecurity, are design properties that are hard to retrofit. Three articles matter most for agents. These summaries follow the text on the AI Act Service Desk on 23 October 2026; check the consolidated text for Omnibus amendments.

Article 12, record keeping. High risk systems must technically allow automatic recording of events over their lifetime, with traceability suited to the purpose. For an agent: tool calls, inputs and outputs, policy decisions and approvals.

Article 14, human oversight. Systems must support effective oversight, including awareness of automation bias, the ability to override or reverse output, and a "stop" button or similar procedure that halts the system safely.

Article 26, deployer obligations. Deployers assign oversight to people with the competence, training and authority to do it, and keep logs under their control for at least six months unless other law says otherwise. EU financial institutions keep them within the documentation their financial services rules require.

More detail: EU AI Act high risk system obligations.

NIST AI RMF and the Generative AI Profile

NIST's AI RMF 1.0, released 26 January 2023, is voluntary and sector neutral, organised into Govern, Map, Measure and Manage. NIST AI 600-1, the Generative AI Profile released 26 July 2024, lists twelve risks generative AI creates or worsens. Two map straight onto agents: information security, which includes attacks such as prompt injection, and value chain and component integration, which NIST describes as untraceable third party components and weak supplier vetting that reduce accountability downstream. An agent built from a hosted model, three MCP servers and an open source framework is that value chain.

NIST's AI RMF page says the framework is being revised under the White House AI Action Plan, and that on 7 April 2026 NIST released a concept note for a profile on trustworthy AI in critical infrastructure. See also NIST's cyber AI profile.

FINRA: technology neutral rules, explicit agent risks

FINRA Regulatory Notice 24-09 (27 June 2024) reminded firms that its rules apply to generative AI like any other technology. The 2026 Annual Regulatory Oversight Report goes further. In summary, it:

• Ties generative AI to existing rules on supervision, communications, recordkeeping and fair dealing, noting that firms relying on AI within a Rule 3110 supervisory system may need to consider the model's integrity, reliability and accuracy.
• Suggests storing prompt and output logs, tracking which model version was used and when, and human review of outputs.
• Lists agent risks: autonomy without human validation, acting beyond intended authority, hard to trace multi-step reasoning, sensitive data mishandling, missing domain knowledge and misaligned reward functions.
• Suggests firms consider how to monitor agent access and data handling, where to place human oversight, how to track agent actions, and how to set guardrails.

That reads almost like a control specification, and each item maps to a step below.

US banking agencies: SR 26-2 leaves agents to your own governance

On 17 April 2026 the Federal Reserve, OCC and FDIC issued revised model risk management guidance. The Federal Reserve's SR 26-2 says it supersedes SR 11-7 and SR 21-8 and is most relevant to banks above $30 billion in assets.

A footnote is easy to misread. It places generative and agentic AI outside the guidance's scope because they are novel and evolving, then says a bank's own risk management and governance should still set controls for anything not covered. There is no safe harbour. Examiners will ask how your general framework handles agents, and the guidance's section on vendor models, which expects firms to understand and validate them even when code and data are proprietary, is a sensible template for the hosted model inside an agent.

HHS: HIPAA access and audit controls, plus Section 1557

The HIPAA Security Rule at 45 CFR 164.312 requires access to ePHI only for persons or software programs granted access rights, mechanisms that record and examine activity, and verification that an entity seeking access is the one claimed. An agent reading a chart is a software program seeking access: it needs its own grant, identity and audit trail, not a borrowed clinician login.

Under Section 1557, 45 CFR 92.210 bars discrimination through patient care decision support tools and requires reasonable efforts to identify and mitigate that risk. A triage agent may fall within it. FDA has its own pathway for AI enabled device software; we could not retrieve FDA's pages for this review, so check fda.gov directly. See also HIPAA and the software supply chain in 2026.

Regulator map at a glance

The agent threat model regulators expect you to know

OWASP's GenAI Security Project published Agentic AI: Threats and Mitigations on 17 February 2025 and the OWASP Top 10 for Agentic Applications for 2026 on 9 December 2025, built with more than 100 contributors.

The central idea is least agency, an extension of least privilege: do not deploy autonomy where it adds no value, because it widens the attack surface, and pair it with observability of what agents do and which tools they call. It is also the cleanest answer to a proportionality question. An agent that drafts while a person sends is a much smaller oversight problem than one that sends on its own.

ASI09 matters doubly because Article 14 names automation bias. An approval prompt clicked through hundreds of times a day is not effective oversight.

Where MCP fits, and what its spec already requires

The Model Context Protocol is an open protocol connecting AI applications to tools and data. MCP servers expose tools; a client inside the agent host calls them. If every tool reaches the agent through one protocol, you have one place for policy, identity and logging. The current specification, revision 2026-07-28, already includes several controls a regulated firm needs.

Human in the loop. The tools section says there should always be "a human in the loop with the ability to deny tool invocations," and recommends showing which tools are exposed, signalling when they run, and confirmation prompts.

Untrusted annotations. Clients must treat tool annotations, such as a read only hint, as untrusted unless they come from trusted servers.

Authorization. A protected HTTP MCP server is an OAuth 2.1 resource server. It must validate that tokens were issued for it as the audience (RFC 8707), accept only tokens for its own resources, and never accept or pass through other tokens. Clients must use PKCE and request only the scopes they need.

Security best practices. The companion guide covers confused deputy attacks on proxies with static client IDs, token passthrough, server side request forgery during discovery, session hijacking, and local server compromise, for which it recommends sandboxing with minimal privileges.

These line up with regulator language. Audience bound tokens give per-agent identity (HIPAA entity authentication, FINRA scope of authority). Confirmation prompts give oversight (Article 14). Tool visibility gives observability (least agency). More: what MCP is and how to secure it, MCP server authorization patterns and scoped credentials per MCP server.

The open source guardrail stack, by layer

"Guardrail" gets used for everything from a content filter to a kill switch. Separate the layers:

Open source matters at each layer because an examiner can read the policy, the protocol and the log schema. A guardrail you cannot inspect is a claim, not a control.

Decision table: how much autonomy each action deserves

Decide per action, not per agent. One agent can carry tools in several tiers.

If you cannot explain why a tool sits in its tier, move it up one.

Step by step: build an agent an examiner can inspect

These steps assume an agent host that speaks MCP. Adapt names and paths.

Step 1: register every agent and what it touches

Record each agent's owner, purpose, users, model and version, MCP servers, tools, data classes and regulatory category (for example "Annex III credit scoring candidate"). Then look for shadow agents in code:

# MCP client configuration files checked into repositories
git ls-files | grep -Ei '(^|/)(\.mcp\.json|mcp\.json|claude_desktop_config\.json|\.cursor/mcp\.json)$'

# MCP SDKs in JavaScript and Python projects
npm ls @modelcontextprotocol/sdk --all 2>/dev/null
pip show mcp 2>/dev/null | grep -E '^(Name|Version)'

Build a dependency inventory for each agent repository and image:

# Results land as JSON in ./output
safeguard scan --dir .
safeguard scan --image registry.example.com/claims-agent:2026.10.1

See running an inventory of MCP servers.

Step 2: classify each tool by impact

Assign every tool a tier from the decision table and record why. Do not rely on the server's own annotations.

Step 3: give each agent its own identity, bound to the tool

Each agent gets its own OAuth client and tokens whose audience is the specific MCP server. When it acts for a user, the token carries that user and a task scoped permission. Check what a server advertises:

# Protected resource metadata (RFC 9728)
curl -s https://mcp.internal.example.com/.well-known/oauth-protected-resource | jq '{resource, authorization_servers, scopes_supported}'

If a server accepts tokens issued for another API, or forwards your token downstream, fail the review.

Step 4: enforce allow, ask and deny in the agent host

Most hosts support permission rules. In Anthropic's Claude Code, for example, rules are evaluated deny first, then ask, then allow; they are enforced by the host rather than the model; and administrators can disable bypass mode in managed settings users cannot override. A claims assistant might use:

{
  "permissions": {
    "allow": ["mcp__claims__get_*", "mcp__policydocs__search"],
    "ask": ["mcp__claims__update_status", "mcp__email__create_draft"],
    "deny": ["mcp__payments__*", "mcp__email__send", "WebFetch", "Bash(curl *)"],
    "disableBypassPermissionsMode": "disable"
  }
}

Payment tools and direct sending are gone, so the agent can only prepare a draft for a person. Keep such rules in centrally managed configuration, not in prompts, which shape what the model tries but not what the host allows.

Step 5: put a policy and logging point in front of every tool call

Host rules are per client. You also need a central gateway or proxy in front of each MCP server that authenticates callers, evaluates policy, runs content checks and writes one record per decision:

{
  "ts": "2026-10-23T09:14:07Z",
  "agent_id": "claims-assistant",
  "agent_version": "2026.10.1",
  "model": "provider/model-name@version",
  "on_behalf_of": "user:j.smith",
  "mcp_server": "claims",
  "tool": "update_status",
  "args_digest": "sha256:9f2c...",
  "policy_version": "claims-policy@41",
  "decision": "ask",
  "approver": "user:a.lee",
  "approved_at": "2026-10-23T09:15:02Z",
  "before": "PENDING_REVIEW",
  "after": "APPROVED"
}

That record answers FINRA's questions on actions and model versions, supports Article 12 traceability and gives a HIPAA auditor the who, what and when. Store full arguments only where your data rules allow.

Step 6: keep logs long enough, and immutable

Use append only storage, with retention set by the strictest applicable rule: at least six months for high risk AI Act deployers, longer where broker-dealer or health record rules apply. Test that you can pull one agent's actions for one customer and date within an hour.

Step 7: design oversight people can perform

Name overseers with authority to stop the agent. Show approvers the exact action and effect. Watch approval rates; 100 percent acceptance signals automation bias. Test the kill switch on a schedule and confirm it revokes credentials.

Step 8: test against the OWASP list before go live

Test goal hijack through documents, tool misuse, privilege escalation through shared credentials, memory poisoning and inter-agent spoofing. Record what failed and what you changed.

Step 9: keep the agent's own software patched

Frameworks, MCP SDKs and servers are ordinary dependencies, which is why OWASP lists ASI04. Gate builds in CI and rebuild when fixes land:

# CI step: scan the agent image before deploy and keep the JSON results as evidence
safeguard scan --image registry.example.com/claims-agent:${GIT_SHA}

Pin MCP server versions, review tool definition changes before upgrading, and record which version each agent ran on each date, so you can answer "which agents were exposed" when an advisory appears.

Where Safeguard fits, honestly

Safeguard is a software supply chain security platform. Most of this guide is your own architecture and governance, and it works without us. Where Safeguard can help, with maturity stated:

• Generally available: SBOM and dependency scanning of agent repositories, container images and SBOM files, with vulnerability matching and reachability analysis; policies, gates and guardrails at build, CI and release; agent identity tracking that shows which agent holds what access at what level; model scanning for LLMs, SLMs, skills and agents; a compliance suite for frameworks, controls and evidence; and autonomous remediation with fix pull requests, shipped switched off.
• Early access: the AI Gateway, an LLM firewall with prompt injection, jailbreak and PII egress guardrails; MCP Guardrail, which issues proxy URLs for MCP servers so tool calls pass through a governed path; and AI-BOM discovery. Evaluate these, and ask which agent hosts and MCP servers are covered in your environment.

Safeguard does not decide whether your use case is high risk, design your oversight, replace your system of record's audit trail or log retention, or change how a hosted model behaves. Using it does not make an agent compliant on its own.

Common mistakes

• Treating the postponed AI Act dates as a reason to wait. Logging, oversight and identity are architecture decisions.
• Reading SR 26-2's scope footnote as an exemption. It says your own governance must cover agents.
• Sharing one service account across agents. You lose attribution and one compromised agent inherits everyone's access.
• Trusting tool annotations. A tool that calls itself read only is not proof.
• Guardrails only in the system prompt. Prompts guide; hosts and gateways enforce.
• Approval prompts nobody reads. Fewer, clearer approvals are better oversight.
• Logging the chat but not the tool calls. Investigators need arguments, credentials and results.
• Ignoring the agent's own dependencies. An unpatched MCP server sits next to your most sensitive data.

FAQ

Can banks use AI agents under current regulations?

Yes. No US or EU rule bans them. Existing supervision, model risk, recordkeeping, consumer protection and data rules apply. US banking agencies left agentic AI out of the April 2026 model risk guidance but expect each bank's broader governance to cover it.

Is an AI agent a high risk AI system under the EU AI Act?

It depends on use. An agent that evaluates creditworthiness, sets credit scores, or prices life or health insurance for individuals falls under Annex III. One that summarises internal documents usually does not.

When do EU AI Act high risk rules apply to financial services?

After the AI Omnibus, the Commission says Annex III obligations, including credit scoring and insurance pricing, apply from 2 December 2027, and Annex I product rules from 2 August 2028. Transparency rules applied from August 2026.

What does FINRA say about AI agents?

FINRA's 2026 oversight report lists agent risks, including autonomy without human validation, acting beyond authority and poor auditability, and suggests monitoring access, human oversight, tracking actions and model versions, and guardrails. Notice 24-09 confirms existing rules apply.

Does HIPAA allow AI agents to access patient records?

HIPAA does not prohibit it, but only authorised persons or software programs may access ePHI, activity must be recorded and examinable, and entities must be authenticated. An agent needs its own grant, identity and audit trail.

Is the Model Context Protocol secure enough for regulated data?

The spec has strong requirements, such as audience bound tokens and a ban on token passthrough, but security depends on implementation. Review each server's authorization, sandbox local servers, log every tool call and keep servers patched.

What is "least agency" in AI security?

OWASP's extension of least privilege for agents: give an agent only the autonomy the task needs. If a person can send the email the agent drafted, do not let the agent send it.

How long should AI agent logs be kept?

High risk AI Act deployers keep logs at least six months unless other law sets a different period. Broker-dealer and healthcare record rules may require longer. Use the strictest that applies.

Do open source models make AI agents easier to audit?

They help with inspection, local hosting and bias testing. But an agent's audit trail mostly comes from tool permissions, identities, policy decisions and logs, which matter whatever the model.

Fix it without waiting for upstream

An agent in a bank or hospital is built from open source frameworks, MCP SDKs, servers and container images, and those components get vulnerabilities like any other code, which is why OWASP lists the agentic supply chain as a top risk. Keeping that layer patched is the part of agent governance most teams can fix today, without waiting on new regulation.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

Neither demo is a full regulated agent deployment. They show two controls from this guide in small, real form: choosing which tools an agent may use, and a person approving each action. To try the CLI, download it from https://safeguard.sh/download and read the setup guide at https://docs.safeguard.sh/docs/cli .

The first demo connects Claude to the Safeguard MCP server with OAuth and chooses which tools Claude can use. That selection step is least agency in practice: the agent gets only the tools a task needs, through OAuth rather than a shared key, and you can review the list.

Click through it at your own pace, or open it full screen.

The second demo is a real Safeguard CLI session. Asked to fix the lodash CVE in a repo, it reads the finding and advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, prints a summary and pushes nothing, then shows /permissions, /model and /scan. Those per-command approvals are the human in the loop pattern the MCP spec and FINRA describe, applied to a coding agent.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• Linux Foundation: Navigating the Agentic AI Guardrails: Why Open Source is the Key to AI in Regulated Industries (Hilary Carter, 20 April 2026)
• Linux Foundation: How Should Enterprises Transition to the Agentic Era? (Irving Wladawsky-Berger, 9 October 2026)
• European Commission: AI Act regulatory framework and application timeline
• European Commission AI Act Service Desk: Article 12, Record-keeping
• European Commission AI Act Service Desk: Article 14, Human oversight
• European Commission AI Act Service Desk: Article 26, Obligations of deployers
• European Commission AI Act Service Desk: Annex III, High-risk AI systems
• NIST: AI Risk Management Framework
• NIST AI 100-1: AI RMF 1.0
• NIST AI 600-1: Generative Artificial Intelligence Profile
• FINRA: 2026 Annual Regulatory Oversight Report, GenAI
• FINRA: Regulatory Notice 24-09
• Federal Reserve: SR 26-2, Revised Guidance on Model Risk Management
• Federal Reserve, OCC and FDIC: Revised Guidance on Model Risk Management (attachment)
• eCFR: 45 CFR 164.312, Technical safeguards
• eCFR: 45 CFR 92.210, Patient care decision support tools
• OWASP GenAI Security Project: Top 10 for Agentic Applications for 2026
• OWASP GenAI Security Project: Agentic AI, Threats and Mitigations
• Model Context Protocol specification 2026-07-28: Tools
• Model Context Protocol specification 2026-07-28: Authorization
• Model Context Protocol: Security best practices
• Anthropic: Claude Code, Configure permissions

Last reviewed: 23 October 2026. AI Act dates, US supervisory guidance and the MCP specification are changing; check the linked primary sources for the latest text.