Building an AI inventory a pharma GxP and EU AI Act review will accept

When quality, legal and the board all ask how the company uses AI, a pharma GRC lead needs one inventory that answers three questions for every AI system: what it is and where it runs, which rules apply to it (GxP, the EU AI Act, Part 11, privacy), and what evidence shows it is controlled. Build that inventory from discovery, not from a survey alone, classify each entry by its context of use and the decision it influences, and keep the evidence beside the entry so an inspector or auditor can follow it.

This post walks that job end to end for a GRC lead at a pharmaceutical company. The first two thirds work with whatever tools you already have. Safeguard enters at "How Safeguard does this", and the matching use-case page is AI Governance.

Key takeaways

• An AI inventory in a regulated pharma company is a GxP record, not a spreadsheet for curiosity. Each entry needs an owner, a context of use, a risk class, the data it touches and a pointer to its validation or assurance evidence.
• In the US, 21 CFR Part 11 and 21 CFR 211.68 already reach AI that touches regulated records or drug manufacturing. FDA's January 2025 draft AI guidance adds a risk-based credibility framework built around context of use.
• In the EU, the AI Act sorts systems into prohibited, high-risk, transparency and minimal tiers. The AI Omnibus (Regulation (EU) 2026/1744) moved the high-risk dates to 2 December 2027 for Annex III systems and 2 August 2028 for AI in Annex I products such as medical devices. The draft EU GMP Annex 22 keeps generative AI and large language models out of critical GMP applications.
• Surveys miss what people actually run. Discover first, then classify.
• Safeguard capabilities covered here, all GA: AI-BOM discovery (GA), AI agent discovery (GA), model artifact scanning in AI-SPM (GA), agent identity (GA), the AI Gateway (GA), policies and gates (GA), and the Compliance and GRC suite (GA) with framework readiness, policy templates and evidence.
• Some things stay your job: deciding the context of use and risk class, validating GxP systems, signing records, answering the regulator and deciding what the company will and will not use AI for.

The situation

You run governance, risk and compliance at a drug company, perhaps as Director of GRC or Head of IT Quality and Compliance. You sit between quality, IT, legal and privacy, and you own the evidence when an inspector or auditor asks how something is controlled.

The questions arrive in the same quarter. Quality asks whether the new deviation-summary assistant is a computerised system that needs validation. Legal asks for "a list of every AI system and its risk category" before the next AI Act dates. The EU affiliate's works council asks whether the recruiting tool scores candidates. A manufacturing site runs a vision model on a packaging line. And internal audit writes that the company "has no complete inventory of AI use".

A bad week: you survey every department head and half reply "we don't use AI". Then you learn a commercial team drafts medical information responses with a general chatbot, developers run coding agents with shell access on laptops that hold validated system credentials, and an R&D cloud account has model endpoints nobody registered. The inspection readiness review is in six weeks, and the inspector will ask for the system inventory first.

What you want is one inventory that is built from what actually exists, classified in a way quality and legal both accept, and tied to evidence that survives an inspection.

The rules and standards behind it

No single rule is called "AI inventory", but together the sources below require you to know what AI you run and control it in proportion to risk. It is not legal, regulatory or audit advice; confirm how each applies to your products, sites and markets with your own counsel and quality leadership.

FDA: Part 11 and drug manufacturing equipment rules

21 CFR Part 11 sets the conditions under which FDA treats electronic records and electronic signatures as trustworthy. Section 11.10 lists the controls for closed systems, including validation of systems to ensure accuracy and reliability (11.10(a)), the ability to generate accurate and complete copies of records (11.10(b)), limiting system access to authorised individuals (11.10(d)), and secure, computer-generated, time-stamped audit trails (11.10(e)). An AI system that creates, modifies or maintains a record required by a predicate rule falls under the same expectations as any other computerised system.

Part 11's scope in section 11.1(b) is broad: it covers records in electronic form that are created, modified, maintained, archived, retrieved or transmitted under any FDA records requirement.

21 CFR 211.68 says automatic, mechanical and electronic equipment, including computers, used in drug manufacturing must be routinely calibrated, inspected or checked under a written program. Section 211.68(b) adds that changes to master production and control records may be made only by authorised personnel, and that input to and output from computer systems must be checked for accuracy, with the degree and frequency of checking based on the system's complexity and reliability. That last clause reads naturally onto AI: the less predictable the system, the more its outputs need checking. A vision model on a packaging line or a model that recommends batch disposition is equipment under that rule.

FDA: AI for regulatory decision making

In January 2025 FDA published the draft guidance "Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products". It proposes a risk-based credibility assessment framework in seven steps. The first steps are the ones an inventory has to support: define the question of interest, define the context of use for the AI model, and assess model risk, which the draft describes as a combination of model influence (how much the AI output contributes to the decision) and decision consequence (what happens if the decision is wrong). The draft covers AI used to produce information or data that supports regulatory decisions about safety, effectiveness or quality, across the nonclinical, clinical, postmarketing and manufacturing phases. It does not cover AI in drug discovery, or AI used for operational efficiencies (its examples include internal workflows, resource allocation and drafting a regulatory submission) that do not affect patient safety, drug quality or the reliability of study results. That boundary is exactly why your inventory needs a field that says which side of the line each entry sits on. It is still a draft (docket FDA-2024-D-4689), so check FDA's guidance page for the current version.

FDA: computer software assurance

FDA finalised its computer software assurance guidance in September 2025, superseding section 6 of the 2002 General Principles of Software Validation, and reissued it on 3 February 2026 as "Computer Software Assurance for Production and Quality Management System Software". Strictly, it is written for medical device production and quality management system software. It asks manufacturers to scale assurance effort to the risk a software feature poses to product quality and patient safety, rather than documenting everything to the same depth. Many pharma quality teams borrow its thinking for GxP systems. For AI, the practical consequence is that your inventory's risk field drives how much assurance evidence each entry needs.

The EU AI Act

Regulation (EU) 2024/1689, the AI Act, entered into force on 1 August 2024 and applies in stages. Chapters I and II, which hold the AI literacy article and the prohibited practices, have applied since 2 February 2025. The chapter on general-purpose AI models has applied since 2 August 2025. The general application date was 2 August 2026, and Article 50's transparency duties for systems that interact with people or generate content apply from then.

The high-risk dates moved this year. The AI Omnibus, Regulation (EU) 2026/1744, was published in the Official Journal on 24 July 2026 and entered into force on 27 July 2026. Under Article 113 as amended, the high-risk rules apply from 2 December 2027 for systems classified high-risk under Article 6(2) and Annex III, and from 2 August 2028 for systems under Article 6(1) and Annex I, which includes AI in medical devices and in vitro diagnostics.

Three parts matter most for a pharma inventory:

• Roles. Article 3 defines a provider (who develops a system or has it developed and places it on the market or puts it into service under its own name) and a deployer (who uses a system under its authority). A pharma company is usually a deployer of bought tools and sometimes a provider of systems it builds and ships, for example AI inside a companion diagnostic or a patient support app.
• Risk tiers. Article 6 and Annex III define high-risk systems. For a pharma company the most common high-risk entries are not in the lab; they are in HR (Annex III point 4, recruitment and worker management) and in any AI that is part of a regulated medical device. Article 50 adds transparency duties for systems that interact with people or generate content.
• Deployer duties. Article 26 requires deployers of high-risk systems to take technical and organisational measures to use them according to the provider's instructions for use, assign human oversight to people with the competence and authority to exercise it, ensure input data under their control is relevant, monitor operation, and keep the logs the system generates for at least six months unless other Union or national law says otherwise.
• AI literacy. Article 4, as amended by the Omnibus, asks providers and deployers to take measures to support the development of AI literacy. Your training records belong in the inventory's evidence.

EU GMP Annex 11 and draft Annex 22

EU GMP Annex 11 covers computerised systems used in GMP activities. On 7 July 2025 the European Commission opened a stakeholder consultation, which closed on 7 October 2025, on a revised Chapter 4 (documentation), a revised Annex 11 and a new Annex 22 on artificial intelligence, all drafted by the EMA GMDP Inspectors Working Group with PIC/S.

Draft Annex 22 applies to computerised systems used in manufacturing medicinal products and active substances where AI models are used in critical applications with a direct impact on patient safety, product quality or data integrity. Its scope section is unusually direct:

• It covers static models only. Dynamic models that keep learning in use should not be used in critical GMP applications.
• It covers models with deterministic output. Models whose output can differ for identical inputs should not be used in critical GMP applications.
• It therefore does not apply to generative AI and large language models, which should not be used in critical GMP applications. In non-critical uses, qualified people must be responsible for checking the output, a human in the loop.

The rest reads like an inventory specification: an intended use approved by a process subject matter expert before acceptance testing, acceptance criteria at least as good as the process the model replaces, representative test data kept independent of training data under access control and audit trail, and recorded feature attribution during testing. Treat it as a draft, but it tells you which fields quality will ask your inventory to hold.

EMA's reflection paper and the NIST AI RMF

EMA's reflection paper on AI in the medicinal product lifecycle, first published on 30 September 2024, sets out the agency's current thinking on using AI to support the safe and effective development, regulation and use of medicines.

The NIST AI Risk Management Framework (AI RMF 1.0, released 26 January 2023) is voluntary, but it gives US companies a shared vocabulary. GOVERN 1.6 reads: "Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities." NIST says the framework is being revised as part of the White House AI Action Plan, so check for an update before you map controls to it. ISO/IEC 42001:2023 sets requirements for an AI management system and is increasingly what customers and partners ask about in questionnaires.

How teams do it today and where it breaks

Most pharma companies start with a survey, a spreadsheet and a committee. The committee is useful. The survey and the spreadsheet break in predictable places.

The survey asks the wrong question. "Do you use AI?" gets a no from teams using a translation tool, an eQMS summariser or a coding assistant, because they do not think of those as AI projects.

The spreadsheet has no source. Each row is what someone said. Nothing links the row to the repository, the cloud account, the SaaS tenant or the laptop where the system actually runs. When an inspector asks "how do you know this is complete?", the honest answer is "we asked".

Classification happens once. A team registers a literature-search assistant as "no GxP impact". Six months later the same assistant drafts sections of an aggregate safety report. Nothing triggers a review.

Agents are invisible. Coding agents run as the user, with the user's access, and rarely appear in any inventory.

Step by step, with or without Safeguard

The steps below work with any tooling. Where a tool helps, the step says what kind of tool, not which product.

1. Agree the definition and the scope before you count anything

Write down what counts as an AI system for your inventory. A practical definition for pharma: any software component that uses a trained model to produce predictions, classifications, recommendations, content or decisions, whether you built it, bought it, or it arrived as a feature of something you already use. Align it with the AI Act's Article 3 definition so legal accepts it, and with your computerised system inventory so quality accepts it.

Scope it to every legal entity, site and function, including HR, commercial and medical affairs.

2. Discover before you survey

Run discovery against the places AI actually shows up, then use the survey to fill gaps and confirm ownership.

3. Record the same fields for every entry

Use one template. The fields below map to what FDA's credibility framework, the AI Act and draft Annex 22 will each ask for.

4. Classify with a decision table

Use a short table that quality and legal both sign. One version:

5. Check the components, not just the system

An AI system is software. Its libraries have vulnerabilities and licences like any other software, and model weights and prompts are components too. Record them in a bill of materials, ideally in a standard format such as the CycloneDX machine learning extension or the SPDX 3.0 AI profile, so the inventory entry can point to something a machine can read. Scan model files before they are loaded, especially formats that can execute code on load.

6. Tie evidence to each entry

For every entry with GxP impact or an AI Act obligation, link the evidence: the validation package or assurance record, the human oversight procedure, the training record for AI literacy, the log retention setting and the last periodic review. An entry without evidence is a finding waiting to happen.

7. Make the inventory live

Set triggers that reopen an entry: a new model version, a new data source, a new use of the output, a new market, a vendor change notice. Re-run discovery on a schedule and reconcile it with the inventory. Anything discovery finds that the inventory lacks is either a new entry or shadow AI to address.

Illustrative scenario: a composite drug maker builds its first complete AI inventory

This is a composite example, not a customer. The company, the people and every number below are invented example inputs to show the workflow.

"Harbor Ridge Pharma" is a US drug maker with manufacturing sites in the US and the EU. Its GRC lead, the inventory owner, has six weeks before an inspection readiness review and an audit finding on AI inventory to close.

Week one: scope. The inventory owner agrees the AI system definition with quality and legal and adopts the field template from step 3. Example input: the GxP computerised system inventory holds around 300 systems, and nobody knows how many use AI.

Week two: discovery. Engineering connects the company's main code hosts and its R&D and manufacturing cloud accounts to a discovery tool. Discovery finds AI SDK imports in a set of repositories, managed model endpoints in the R&D cloud account, two vector databases, and a handful of model weight files committed to a data science repository. IT runs a desktop discovery pilot with a volunteer group of developers and finds coding agents with shell and network access, connected to MCP servers for the issue tracker and the document management system.

Week three: classification. The inventory owner runs the decision table with system owners. The packaging line vision model is direct GxP impact and already validated as part of the line, but its model version is not under change control. The deviation summary assistant is indirect impact: it drafts, a QA reviewer approves in the eQMS. A recruiting tool used by the EU affiliate is flagged for legal review as an Annex III candidate. Commercial use of a general chatbot for medical information drafts is stopped pending a controlled process.

Week four: controls. Model files are scanned before reuse, CI policies start in warn mode on AI repositories, LLM applications go behind a guardrail in monitor mode, and agents with shell access near validated systems are reviewed.

Weeks five and six: evidence. Each direct and indirect entry gets its evidence links, and the inventory owner writes a one page summary for the readiness review: how the inventory was built, how it stays current, and open items with owners and dates.

The scenario shows the order of work, not results, because results depend on your estate.

How Safeguard does this

Safeguard covers discovery, components and controls, and keeps compliance evidence in one place. Each step links its docs page; where the docs show no screen or command, the step describes the outcome.

1. Turn on the frameworks you answer to

In the Compliance and GRC suite (GA), open Compliance › Manage, then the Frameworks tab, find a framework by region, industry or type, switch it on and select Save. It then appears on Compliance › Frameworks, Controls and Overview. The library includes the EU AI Act, the NIST AI RMF and FDA 21 CFR Part 11. Safeguard's docs are clear that choosing a framework does not certify compliance. (Compliance frameworks)

Each framework gets a readiness score: the share of its in-scope requirements that pass. A requirement passes when a scan finds it met, or when a person attests it and names the evidence. A requirement a person scopes out with a written justification leaves the score. The docs state that the score is Safeguard's assessment of what it can see, not an audit opinion. The score shows where evidence is missing; the attestations carry your judgement.

2. Start the AI policy from a template

Every framework in the library carries a set of policy templates, including an AI & Responsible Use Policy. Open the framework on Compliance › Frameworks, then its Policies tab, select New policy and choose Start from template. Edit it to your acceptable use rules and GxP impact classes. (Compliance frameworks)

3. Build the AI-BOM from code, cloud and SaaS

AI-BOM discovery (GA) inventories AI assets across three surfaces. In code it finds imports of AI SDKs, model weight files, prompt templates, notebooks and MCP server configuration. In cloud it finds managed inference endpoints in SageMaker, Bedrock and Vertex AI through your cloud connectors. In connected SaaS it surfaces unsanctioned AI services. Vector databases are recorded as vector stores and agent definitions as agents. (AI-BOM Discovery)

The assets for a product are assembled into an AI-BOM, a tenant and organisation scoped snapshot of your AI footprint at a point in time, emitted in the CycloneDX ML-BOM extension format and the SPDX 3.0 AI profile. Per model, the docs list identity (name, version, architecture, parameter count, quantisation), where the weights came from and their hash, fine-tune lineage, dataset references, dependencies and deployment targets. That covers most of the "what is it and where does it run" fields. (AI-BOM & Model Security)

Discovery runs through the standard scan flow once an administrator enables it. Shadow AI services surface for governance review, where your acceptable use policy decides what happens next.

4. See the coding agents on laptops

AI agent discovery (GA) runs inside the Safeguard CLI and Desktop app. For supported agents it reports, per device and person, which agents are installed, what each is allowed to do (shell commands, file edits, network access, auto-approved tools, permission mode), the MCP servers each connects to, the skills and plugins installed, local model files, and the credentials an agent refers to, by name and kind only. (AI agent discovery)

Safeguard enables the feature for your organisation, an administrator turns it on and sets the collection policy, and each person allows it on their own computer. In the CLI, the docs show:

safeguard aidr collect status    # your organisation's setting, your choice, the last delivery
safeguard aidr collect on        # read the notice and allow it
safeguard aidr collect show      # print exactly what the next run would send (nothing is sent)

Results appear under Assets › Inventory › Asset discovery, with tabs for Agents, Models, Skills & plugins, MCP servers and Devices. The privacy defaults matter in a company with works councils: by default prompts, responses, full tool inputs, tool output, file contents and credential values are not collected, file paths are shortened, and shell commands are recorded by program name only. Only the agents the docs list are detected.

5. Record which agent holds what access

Agent identity (GA) tracks which agent holds what access, at what level. That answers the reviewer's question about any agent near a validated system: what could it reach, and with whose authority.

6. Scan model files before anyone loads them

Model artifact scanning in AI-SPM (GA) analyses model files statically, without loading them, across Pickle, PyTorch, safetensors, GGUF and ONNX formats. For Pickle and PyTorch files it flags code-execution steps that would run when the file is deserialised, mapped to CWE-502 and CWE-94; for ONNX it also flags external-data references that escape the model directory, mapped to CWE-22. Each finding names the artifact, the format, the risk and the CWE, and lands in the unified findings model. (AI-SPM)

7. Put a guardrail in front of LLM applications

The AI Gateway (GA) sits between an LLM application and its model provider, as an inline proxy or a thin shim around the LLM client, and evaluates prompts, responses and tool calls against guardrails: direct and indirect prompt injection, jailbreak patterns, personal data leaving in prompts or responses, and output format. Monitor mode is the default everywhere: every interaction is evaluated, a guardrail event is recorded and traffic passes unchanged. Enforce mode, which redacts or blocks, needs an explicit per-tenant opt-in. On a gateway error or timeout it fails open by default, so the application keeps running and an alert is raised. (AI Gateway)

For a GxP application, monitor mode is the natural first step: the guardrail events are tenant scoped and audited, which gives you a record of what the gateway saw before anyone decides whether enforcement belongs inside a validated process.

8. Gate the code that makes up AI systems

Policies and gates (GA) evaluate scans and decide what a failed policy does to a gate. Create a policy under Settings › Policies, define conditions with the visual builder or YAML, set severity and action, then save and enable it. Each policy's action is one of three:

When a team needs an exception, it requests one with the policy, the finding, a business justification and an expiry date; approvers decide and the audit trail records it. With policy outcomes on for your workspace, the docs show this CI command failing a job when the gate fails (the gate name is a placeholder; this snippet was not run for this post):

safeguard ci scan --wait --fail-on-gate GATE_NAME

(Policies & Gates)

9. Attest what a scan cannot see

Many inventory controls are not visible to any scanner: the human oversight procedure, the AI literacy training record, the validation summary. In the Compliance and GRC suite these pass when a person attests them and names the evidence, and they stay in the readiness score as not yet passing until someone does.

The approvals model is built for this kind of record. (Approvals)

• Evidence. On Compliance › Evidence, evidence starts as submitted and a compliance reviewer chooses Accept or Reject, stamped with who, when and why. Changing accepted evidence creates a new version that goes back to submitted, so an acceptance always covers the version the reviewer saw.
• Policies. On Compliance › Policies, a policy goes to review with Submit for review. Editing an approved policy sends the new version to review while the approved one stays in force, and a reviewer can launch an acknowledgement campaign so staff acknowledge a specific version. That fits an AI acceptable use policy and the training evidence Article 4 calls for.
• Decisions stay with people. Safeguard's AI assistant and MCP clients can draft and submit, but never accept, approve, launch or acknowledge. For a GxP audience that separation is worth stating in your procedure.

10. Keep the model choice inside your rules

Safeguard's AI modes are set per tenant (GA): Zero AI runs with no LLM anywhere, bring your own model uses one you choose, and on-prem or air-gapped deployment keeps models in your environment. Customer code, SBOMs and telemetry are not used to train shared models.

Requirement, what Safeguard does, what you still own

What to measure

Use these as a checklist. They are measures to set up, not results anyone should promise.

• Share of inventory entries with a discovery source (repo, cloud account, SaaS tenant, device) rather than a survey answer only.
• Number of discovered AI assets with no inventory entry, tracked to zero over time.
• Share of entries with a completed context of use, GxP impact and AI Act tier.
• Share of direct and indirect GxP entries with linked validation or assurance evidence.
• Model files scanned before first load, as a share of model files discovered.
• Coding agents with shell access on machines that hold validated system credentials.
• Readiness score for each AI-related framework, read with the count of requirements still waiting for evidence.

What Safeguard does not do

• It does not decide your context of use or risk class. Safeguard records what exists and where; whether a model's output influences a batch release decision is a judgement your quality unit makes and documents.
• It does not validate your GxP systems. Validation and computer software assurance of an AI-enabled system remain your quality system's work. Safeguard's scan and policy results can be evidence inside that work, not a substitute for it.
• It does not make AI Act determinations. Whether you are a provider or a deployer and whether a system is high-risk under Annex III is a legal question. The readiness score is not an audit opinion or a certification.
• It does not see everything. AI-BOM discovery covers what you connect. Agent discovery covers only the agents its docs list, on computers running the Safeguard CLI or Desktop app with the person's consent. AI inside an unconnected vendor system stays a survey and questionnaire item.
• It does not read prompts by default. Agent discovery leaves out prompts, responses and file contents unless prompt capture is enabled at every level and the person agrees. If you need content-level review, that is a separate decision with privacy and works council input.
• It does not judge scientific or clinical validity. Safeguard does not evaluate whether a model's predictions about assays, compounds or patients are correct. That is a scientific and statistical question for your own experts.
• It does not sign your records. Part 11 electronic signatures on regulated records stay in your validated systems.

Common mistakes

• Starting with the survey. A survey without discovery measures awareness, not usage. Discover first, then ask owners to confirm.
• One inventory for AI, another for GxP systems. Keep one register or a hard link between them, or the same model will have two owners and two risk classes.
• Classifying by technology instead of use. "LLM" is not a risk class. The same model can be harmless in an internal search tool and direct impact when it drafts a regulated record.
• Forgetting HR and commercial. The AI Act's most likely high-risk entries for a pharma company are often in recruitment, not the lab.
• Treating model weights as data. Some formats execute code on load. Scan them like software.
• Writing exceptions without expiry. Every exception needs a date and a reason, or it becomes the policy.

FAQ

Does the EU AI Act apply to a US pharma company?

It can. The Act applies to providers placing AI systems on the EU market and to deployers located in the EU, and in some cases to providers and deployers outside the EU where the output is used in the EU. A US company with EU affiliates or EU patients should ask counsel to map its exposure.

Is AI in drug manufacturing high-risk under the AI Act?

Not automatically. High-risk status comes from Article 6, Annex I (AI in products such as medical devices) and Annex III (listed use areas such as recruitment). Manufacturing AI is governed mainly by GMP rules, including EU GMP Annex 11 and the draft Annex 22.

What does FDA expect for AI that supports regulatory decisions?

FDA's January 2025 draft guidance proposes a seven-step, risk-based credibility assessment that starts with the question of interest and the context of use, then assesses model risk as a combination of model influence and decision consequence. It is a draft, so check FDA's site for the current version.

Does Part 11 apply to AI tools?

When an AI system creates, modifies, maintains, archives, retrieves or transmits a record required by an FDA predicate rule, Part 11's controls apply to it, including validation, access limits and audit trails.

How does Safeguard find AI that nobody declared?

AI-BOM discovery looks for AI SDK imports and model files in code, managed model endpoints in connected cloud accounts and unsanctioned AI services in connected SaaS. AI agent discovery reports coding agents, their MCP servers and local models on consenting users' computers.

Can Safeguard tell us whether a system is high-risk?

No. It tracks the EU AI Act as a framework with a readiness score and stores your attestations and evidence. The classification is a legal and quality decision you make.

How often should the AI inventory be reviewed?

On a schedule and on triggers. Re-run discovery regularly, and reopen an entry when the model version, data source, use of the output, market or vendor changes. Direct GxP entries also follow your periodic review cycle for validated systems.

Does Safeguard collect employees' prompts?

Not by default. AI agent discovery collects tool names, timings, models, token counts, shortened file paths and program names. Prompts are collected only if Safeguard enables prompt capture, an administrator turns it on and the person agrees, and even then they are masked and truncated.

Related reading

• Pharma software validation and supply chain security
• Shadow AI: discover and govern unsanctioned AI tools
• AI-BOM and ML-BOM: the state of the standards
• How the EU AI Act and ISO 42001 interact
• Securing MCP servers when every team connects agents to internal tools

AI systems are software, and the open source libraries inside them, from model serving frameworks to vector database clients, carry vulnerabilities like any other application. Inside a validated GxP process every fix also passes through change control, so the block below explains how Safeguard shortens the gap for those components.

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation categories include dependencies, licenses, security posture and findings, and each one is set separately. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds.

3. Gold Registry: start from components that are already fixed

Gold Registry provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs and a completed license check, with signed build provenance (signed with Safeguard's key and verifiable with cosign, SLSA Build Level 3 as the design target) and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. Ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo connects Claude to the Safeguard MCP server with OAuth and shows choosing which tools Claude can use. An assistant gets the tools you decide, and that choice belongs in the agent's inventory entry.

Click through it at your own pace, or open it full screen.

The second demo sets up the Safeguard local runner on your own machine, approves it with its device code, then scans from the terminal. AI agent discovery runs while the runner or Desktop app is running, so a pilot group starts with this setup.

Click through it at your own pace, or open it full screen.

The third demo is a real CLI session. You install the CLI, run safeguard to open the interactive terminal and ask it to fix the lodash CVE in a repo. It reads the finding and advisory, asks before any command that changes something, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, prints a summary and pushes nothing. An agent proposes and a person approves, the same pattern your change control expects.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• eCFR: 21 CFR 11.1, Scope
• eCFR: 21 CFR 11.10, Controls for closed systems
• eCFR: 21 CFR 211.68, Automatic, mechanical, and electronic equipment
• FDA: Considerations for the Use of AI to Support Regulatory Decision-Making for Drug and Biological Products (draft, January 2025)
• FDA: Computer Software Assurance for Production and Quality Management System Software
• Federal Register: CSA final guidance availability notice, 24 September 2025
• European Commission: consultation on EU GMP Chapter 4, Annex 11 and new Annex 22
• European Commission: draft Annex 22, Artificial Intelligence (PDF)
• EMA: Reflection paper on the use of AI in the medicinal product lifecycle
• AI Act Explorer: Article 4, AI literacy (as amended)
• AI Act Explorer: Article 26, Obligations of deployers of high-risk AI systems
• AI Act Explorer: Article 113, Entry into force and application (as amended)
• Future of Privacy Forum: EU AI Act implementation timeline after the AI Omnibus (PDF)
• NIST: AI Risk Management Framework
• NIST AI 100-1: AI RMF 1.0 (PDF)
• Safeguard docs: AI-BOM Discovery
• Safeguard docs: AI-BOM & Model Security
• Safeguard docs: AI agent discovery
• Safeguard docs: AI Security Posture Management (AI-SPM)
• Safeguard docs: AI Gateway (LLM Firewall)
• Safeguard docs: Compliance frameworks
• Safeguard docs: All frameworks
• Safeguard docs: Approvals
• Safeguard docs: Policies & Gates
• Safeguard: AI Governance use case

To try it, download Safeguard or start with the documentation.

Last reviewed: 25 October 2026. Product capabilities change; check docs.safeguard.sh for the current state.