How Do You Ship AI-Written Code You Don't Fully Understand?

You ship it by deciding, before the pull request exists, which code a human must understand line by line and which code may ship on evidence instead. Authentication, permissions, money, cryptography, data deletion and deploy pipelines need a person who can explain every line. Everything else can ship on tests that pin behaviour, a reviewed dependency diff and a clean scan, with a named owner who accepts the result.

A team that treats every agent written change the same way either drowns in review or quietly stops reviewing. This post is the practical playbook: how to tier the risk, what to check in the code the agent imported, what a second model can and cannot catch, what a pull request should carry before merge, and who owns the bug when nobody read the code.

Key takeaways

• Not understanding code is a maintainability problem first. It becomes a security problem the moment that code sits on a trust boundary: identity, authorisation, money, secrets, untrusted input, infrastructure or the build itself.
• Write the risk tiers down. Map paths in the repository to tiers with a CODEOWNERS file and branch rules, so the tier is enforced by the platform and not by memory.
• Most of what an agent "writes" is code it imports. Review the dependency diff, including transitive packages in the lockfile, as seriously as the code diff.
• A second model reviewing the pull request is a useful extra filter. It does not replace a human who understands the intent, because both models can share the same blind spot and neither knows what the change was supposed to do.
• Make the agent explain its change in a fixed format, and make the reviewer check the explanation against the diff rather than read the diff cold.
• Every agent written pull request should carry evidence: tests that would fail without the change, a dependency diff, a scan result and a rollback note.
• Ownership does not move to the tool. The person who approves the merge and the team that owns the service own the bug, the same as with any other code.
• When the vulnerable code is in a dependency, you do not have to wait for the maintainer. Fix pull requests, a known safe version lookup and hardened builds close most of that gap.

Why so many teams are asking this now

In October 2026 an Ask HN thread titled "How to deal with shipping AI-written code you don't understand?" became one of the most discussed posts on Hacker News. When we fetched it, it showed 614 points and 748 comments. An earlier, much smaller thread asked how to make engineers understand the code they ship at all. The two threads circle the same answers, and they are worth summarising because they reflect what working engineers actually do.

The recurring answers: keep design in human hands and let the model fill in function bodies; agree the approach with the model before it writes code and keep the commit step human; scale scrutiny with risk; and focus review on whether the tests capture the intended behaviour, because line by line reading does not scale to agent volume. The smaller thread added trust tiers, where critical code cannot import from lower trust code, and warned that understanding cannot be mandated by metrics.

None of that is new engineering. It is the same discipline teams used for code from contractors, acquisitions and open source, applied to a source that produces far more code, far faster, with confident explanations that may or may not match what the code does.

Is "I don't understand it" a security problem or a maintainability problem?

Both, but not at the same time and not in the same places.

When it is only a maintainability problem

Code you do not understand is expensive to change. The next fix takes longer and the next agent session works from a mental model nobody on the team holds. As several Hacker News commenters put it, reading diffs is partly how a developer keeps the accurate picture of the system needed to steer future agent sessions.

For a dashboard layout, a CSV export or a retry wrapper, that is the whole risk: if it is wrong, a test fails or a user complains, and the fix is cheap. Tests and observability are a reasonable substitute for line by line understanding there.

Where it becomes a security problem

It turns into a security problem when the code you do not understand makes a decision an attacker would like to influence. A useful test is to ask whether the code does any of the following:

• Decides who someone is or what they may do (authentication, session handling, authorisation checks, tenant isolation).
• Moves or records money, credits, entitlements or quotas.
• Handles secrets, keys, tokens or cryptographic operations.
• Parses or acts on untrusted input at a trust boundary (request handlers, file uploads, deserialisation, template rendering, query construction).
• Changes infrastructure or network exposure (infrastructure as code, firewall rules, bucket policies, ingress).
• Changes how the software is built or deployed (CI workflows, build scripts, Dockerfiles, release jobs).
• Deletes or migrates data.
• Adds, removes or upgrades a dependency.

If the answer is yes, "the tests pass" is not enough. Tests check the behaviour you thought to test. Security bugs usually live in behaviour nobody thought to test: the missing tenant filter on one query, the comparison that is not constant time, the error path that logs a token, the new transitive package with a known exploited flaw.

OWASP's 2025 list for large language model applications puts this under its misinformation entry, which covers both hallucination and overreliance. It notes that models can suggest "insecure or non-existent code libraries" and that developers can integrate unsafe suggestions without verifying them. Its main mitigation is human oversight and cross checking against trusted sources, which is exactly the tiered approach below.

The point where one becomes the other

The boundary is not fixed. A utility function is maintainability risk until a request handler starts calling it with user input. That is why tiering by path alone is not enough: the reviewer of a tier 3 change has to look at what the change calls, not only the lines it touches. If a payment handler now calls a new agent written helper, that helper has just joined tier 3.

Which code must a human understand line by line?

The table below is a starting point. Adjust it to your system, then write it down where the tooling can enforce it.

Two rules make the table work.

First, tier 3 includes the files that change how other code is trusted. A CI workflow, a Dockerfile and a lockfile are not "config"; they decide what runs in production. Agents edit them often, and a reviewer who skims them misses the most consequential change in the pull request.

Second, the highest tier touched wins. A pull request that changes one CSS file and one line of an authorisation check is a tier 3 pull request. Splitting it is the agent's job, not the reviewer's.

What "understand line by line" actually means

For tier 3 it means the reviewer can answer, without asking the agent: what does each branch do, what happens on each error path, what inputs reach this code and from where, what the code logs, and what changes if a dependency it calls behaves differently. If the reviewer cannot answer, the change waits, gets simplified, or gets rewritten by a human. "The model explained it to me" is not an answer, because the explanation and the code can disagree.

For tier 2 it means understanding the contract: given these inputs, the code produces these outputs and these side effects, and the tests prove it. The reviewer does not have to trace every helper, but they have to be able to say what the change does to the system.

What a risk-tiered review policy looks like in practice

A policy that exists only in a wiki will be ignored by week three. Put it in the repository.

Step 1: map paths to tiers with CODEOWNERS

Your code host can require approval from the owners of the files a pull request touches. On GitHub, a CODEOWNERS file in .github/, the repository root or docs/ maps path patterns to users or teams, the last matching pattern wins, and a branch protection rule or ruleset can require code owner review before merge. GitHub's documentation also recommends giving the CODEOWNERS file itself an owner so nobody can quietly remove a rule.

# .github/CODEOWNERS
# Tier 1 and 2 default: the service team
*                               @acme/payments-service

# Tier 3: identity, money, crypto
/src/auth/                      @acme/security-reviewers
/src/billing/                   @acme/billing-owners @acme/security-reviewers
/src/crypto/                    @acme/security-reviewers

# Tier 3: build, deploy and infrastructure
/.github/workflows/             @acme/platform-owners
/Dockerfile                     @acme/platform-owners
/infra/                         @acme/platform-owners

# Tier 3: dependencies
/package.json                   @acme/dependency-owners
/package-lock.json              @acme/dependency-owners
/requirements*.txt              @acme/dependency-owners
/go.mod                         @acme/dependency-owners
/go.sum                         @acme/dependency-owners
/pom.xml                        @acme/dependency-owners

# Protect the policy itself
/.github/CODEOWNERS             @acme/security-reviewers

Two limits from the documentation are worth knowing. Owners need write access to the repository or they are not requested, and when a line lists several owners, approval from any one of them satisfies the rule. If you want two independent approvals on billing code, require two approvals in the branch rule as well.

Step 2: label agent pull requests and limit their size

A label or a template checkbox marking agent authored pull requests lets the extra checks run and lets you measure outcomes later. Then cap size on critical paths: a tier 3 change of 40 lines can be understood, one of 900 lines cannot.

Step 3: decide what the agent may do without asking

Both major agent products let you control this, and the defaults differ by surface and version, so check your own settings rather than assuming.

In Claude Code, Anthropic's security documentation describes a permission mode per session. Auto mode, which it lists as the built in starting mode for interactive terminal and VS Code sessions, uses a separate classifier model to review actions and block those it judges unsafe. Manual mode starts with read only permissions and asks before editing files, running tests or executing commands. Organisations can enforce settings with managed settings and turn auto mode off. The same page states plainly: "You're responsible for reviewing proposed code and commands for safety before approval."

In Codex, OpenAI's agent approvals documentation describes two layers: a sandbox mode (read-only, workspace-write or danger-full-access) and an approval policy (on-request or never, plus a granular option). It recommends the Auto preset, workspace-write with on-request, for version controlled folders and read-only for folders that are not under version control. Network access is off by default in workspace-write unless you enable it. An optional setting routes approval requests to a reviewer agent instead of to you.

For tier 3 work, a conservative configuration makes sense: the agent can read and propose, and every command and edit outside the working directory waits for a person. For tier 1 work, a sandboxed, more autonomous mode is reasonable because the review happens at merge.

Most of the code an agent ships is code it imported

This is the half of the problem that gets the least attention. An agent asked to "add PDF export" may write 80 lines of code and add a package that brings 40 more packages with it. The 80 lines get reviewed. The 40 packages, and the tens of thousands of lines inside them, often do not.

The Open Source Security Foundation's guide to writing security focused instructions for AI code assistants, published on 1 August 2025, spends a large part of its length on this. It tells assistants to prefer well known libraries over obscure ones, to install through the official package manager, to specify versions, and to avoid dependencies that may be malicious or hallucinated. It cites a study in which 19.7 percent of packages proposed by models did not exist, which is the opening that "slopsquatting" attacks exploit: someone registers the name a model tends to invent.

NIST's Secure Software Development Framework, SP 800-218 version 1.1, already covers this ground for human written code. Its practice PW.4 is about reusing existing, well secured software and reviewing third party components before including them, and PW.7 is about reviewing and analysing human readable code for vulnerabilities. Neither practice gets weaker because an agent chose the component.

What to check in every dependency change

For every package that is new, upgraded or newly pulled in transitively, a reviewer should be able to answer:

1. Does the package exist under exactly this name, and is it the one the agent meant? Check the registry page, the repository link and the publisher.
2. Was a new direct dependency needed at all, or does the standard library or an existing dependency already do the job?
3. Which version was chosen, and is it pinned in the lockfile?
4. Does this version, or anything it pulls in, have known vulnerabilities, and are any of them known to be exploited?
5. Is the package or version end of life or unmaintained?
6. Is the licence acceptable for how you ship?
7. Does it run install scripts?
8. Did the lockfile change in ways the manifest change does not explain?

The last question catches a common agent habit: running a fresh install that silently re-resolves half the tree, so a one line manifest change arrives with hundreds of lockfile lines.

Step by step: review the dependency diff by ecosystem

These commands run on the pull request branch against your main branch. Adjust branch names to your setup.

npm

# What changed in the manifest and lockfile
git diff origin/main...HEAD -- package.json package-lock.json --stat
git diff origin/main...HEAD -- package.json

# Packages added or changed in the lockfile (names only)
git diff origin/main...HEAD -- package-lock.json \
  | grep -E '^\+\s+"node_modules/' | sed 's/[+": {]//g' | sort -u

# Why is a package in the tree, and who pulled it in
npm explain some-package

# When was each version published (very new versions deserve a second look)
npm view some-package time --json | tail -n 5

# Clean, reproducible install that fails if manifest and lockfile disagree,
# without running install scripts during review
npm ci --ignore-scripts
npm audit --omit=dev

The npm documentation for npm ci explains why it belongs in review and CI: it requires an existing lockfile, exits with an error if package.json and the lockfile disagree instead of updating the lockfile, and never writes to either file. If an agent changed the manifest without updating the lockfile, or the other way round, npm ci fails loudly.

Python with pip

git diff origin/main...HEAD -- 'requirements*.txt' pyproject.toml

# Install only pinned, hashed wheels
pip install --require-hashes --only-binary :all: -r requirements.txt

# See what is installed and what requires what
pip list --format=freeze
pip show some-package

The pip documentation on secure installs describes hash checking mode: every requirement and every dependency must be pinned with == and carry a --hash, otherwise installation fails, and it recommends pairing it with --only-binary :all: to refuse source distributions. A requirements file in this form makes any agent edit visible, because a new package without a hash breaks the install.

Go

git diff origin/main...HEAD -- go.mod go.sum

# Full module list and why a module is present
go list -m all
go mod why -m example.com/some/module

# Confirm go.sum matches what is downloaded
go mod verify

# Known vulnerabilities in code paths the program actually calls
govulncheck ./...

Maven

git diff origin/main...HEAD -- pom.xml

# Full resolved tree, including transitive dependencies
mvn -q dependency:tree -Dverbose

Container images

# Pin base images by digest, not only by tag, so an agent edit to the tag is visible
FROM node:22-bookworm-slim@sha256:REPLACE_WITH_DIGEST

An agent that "fixes the build" by changing a base image tag has changed every operating system package in your image. That is a tier 3 change.

Automate the diff on every pull request

Manual commands are for the reviewer. The same check should also run automatically. GitHub's dependency review shows added, removed and updated dependencies in a pull request, including changes to indirect dependencies in lockfiles, with release dates and known vulnerabilities. Its companion action can fail the check, and a required check blocks the merge. The action's README documents inputs such as fail-on-severity, fail-on-scopes, deny-packages and comment-summary-in-pr.

# .github/workflows/dependency-review.yml
name: Dependency review
on: [pull_request]

permissions:
  contents: read
  pull-requests: write

jobs:
  dependency-review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/dependency-review-action@v5
        with:
          fail-on-severity: high
          fail-on-scopes: runtime
          comment-summary-in-pr: always

GitHub notes that the action is available for public repositories and, for private repositories, requires its paid code security features. Whichever tool you use, the requirement is the same: the dependency diff is computed by a machine, posted where the reviewer sees it, and able to block the merge.

Where Safeguard fits in the dependency half

This is the part of the problem Safeguard is built for, and it is worth being precise about what that covers. The Safeguard CLI can scan a repository, a container image or an SBOM file (safeguard scan --dir ., --image and --sbom), building an inventory of direct and transitive dependencies and matching it against vulnerability databases, with results written as JSON. Reachability analysis ranks findings by whether your code actually calls the vulnerable path, so a reviewer looking at an agent's pull request sees the two findings that matter instead of forty that do not. Policies and gates evaluated in the platform can block or warn at merge, and the platform shows which other projects contain the same component and version. When a fixed version exists, Griffin can open a fix pull request with the upgrade and an explanation a person can review. That remediation ships switched off, so your team decides what runs on its own.

For a reviewer who just wants to know whether the version an agent picked is safe, Gold Open Source at gold.safeguard.sh is free, needs no login, and shows affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version for any package or CVE.

Safeguard does not understand your business logic; it will not tell you that an authorisation check is in the wrong place. That stays the human's job. For a deeper look at how agents choose dependencies, see do AI coding agents add vulnerable dependencies.

Can a second model reviewing the pull request replace human understanding?

No. It can catch real problems, and you should probably use one. It cannot stand in for a person who knows what the change was supposed to do.

Both agent makers now ship model based review. Anthropic's Claude Code documentation lists a /security-review command that runs an on demand security pass over the changes on the current branch, and a security guidance plugin that has Claude review and fix vulnerabilities in its own changes during a session. OpenAI's Codex documentation describes an optional auto review setting that routes approval requests to a reviewer agent instead of to the user. These are useful tools. They are also explicitly framed by their makers as additions to human review, not replacements: Anthropic's security page tells users to review all suggested changes before approval, and OpenAI's approvals page tells users to treat Codex suggestions like any other pull request.

What a reviewer model is good at

It spots common vulnerability patterns (string built queries, missing output encoding, unsafe deserialisation, overly broad permissions), notices inconsistencies such as input validated on one path but not another, summarises a large diff so a human knows where to look, and reads the 600th line with the same attention as the first.

What it misses

• Intent. A reviewer model sees what the code does, not what the ticket asked for. If the agent misunderstood the requirement, the code can be internally consistent, well tested and wrong. Only someone who knows the requirement can see that.
• Context outside the diff. Whether this endpoint is internet facing, whether this table holds regulated data, whether this tenant ID can be attacker controlled. Much of that lives in people's heads and in other repositories.
• Correlated blind spots. If the writing model and the reviewing model learned the same insecure pattern, the reviewer may approve it with confidence. Two models agreeing is weaker evidence than it looks.
• Dependency risk. A model reading the diff sees import somelib. It does not know, unless a tool tells it, that the resolved version has a known exploited vulnerability, that the package was published yesterday, or that it is end of life.

How the approaches compare

The practical answer is layers. Let the reviewer model and static analysis do the first pass so the human spends attention where only a human can help.

How do you make the agent explain its change so a reviewer can verify it?

The goal is not a longer pull request description. It is a description structured so each claim can be checked against the diff in a minute.

A pull request template for agent changes

## What this changes
One paragraph, in terms of behaviour a user or caller would notice.

## Why
Link to the ticket. Quote the requirement being met.

## Files and tiers
- src/billing/invoice.ts (tier 3): adds rounding rule for multi currency totals
- src/ui/InvoiceTable.tsx (tier 1): displays the new rounding note

## Dependencies
- Added: none
- Upgraded: none
- Lockfile changes: none
(If any: name, old version, new version, why it was needed, alternatives considered.)

## Trust boundaries touched
Which inputs reach this code, from where, and how they are validated.

## Tests
- Tests that fail without this change: test/billing/rounding.test.ts (3 cases)
- Negative cases: invalid currency code, zero amount, negative amount

## Risks and rollback
What could go wrong in production and how to revert.

## What the agent was unsure about
Anything the agent guessed, assumed or could not verify.

The last section is the most useful: agents will state their assumptions if asked, and that is where review should start.

Put the rules in the agent's instructions file

The OpenSSF guide is built around this idea: put security requirements into the instruction files the assistant reads at the start of every session (for example CLAUDE.md or AGENTS.md). Its headline principle is that the developer is responsible and the assistant assists, and it recommends that instructions be concise, specific and actionable. An example section:

## Rules for changes in this repository
- Never add a new dependency without stating why the standard library or an
  existing dependency cannot do the job. Name the exact package and version.
- Never edit files under src/auth, src/billing, src/crypto, infra or
  .github/workflows in the same pull request as other changes.
- For every change, write or update a test that fails without the change.
  For auth and billing, include negative tests.
- Run the test suite and the dependency scan before proposing a pull request,
  and paste the summary into the pull request description.
- In the pull request, list every assumption you made and anything you could
  not verify.

Ask for a walkthrough, then check it

For a tier 3 change, have the reviewer ask the agent to walk through each branch of the changed function and each error path. Then the reviewer reads the code and checks the walkthrough against it. Disagreements between the explanation and the code are the bugs you are looking for. This is faster than reading cold, and it trains the reviewer's attention on exactly the places the model was least sure about.

Better still, agree the design with the agent before it writes anything, as several Hacker News commenters described, so the human owns the structure. You cannot verify a design you never saw.

What evidence should a pull request carry before merge?

Treat the pull request as a package of evidence, not just a diff. The reviewer's job becomes checking the evidence, which scales better than re-deriving it.

A trick that works well for the first row: in CI, check out the new test files onto the main branch and run them. If they pass on main, the tests do not prove anything about the change. Agents are good at writing tests that pass; this check makes sure they are also tests that mean something.

# Run the PR's new or changed tests against main to confirm they fail there
git fetch origin main
CHANGED_TESTS=$(git diff --name-only --diff-filter=AM origin/main...HEAD | grep -E '\.test\.(js|ts)$')
git worktree add /tmp/main-check origin/main
for f in $CHANGED_TESTS; do
  mkdir -p "/tmp/main-check/$(dirname "$f")" && cp "$f" "/tmp/main-check/$f"
done
cd /tmp/main-check && npm ci --ignore-scripts
npx jest $CHANGED_TESTS && echo "WARNING: new tests pass without the change"

Treat that script as a sketch for your test runner rather than a drop in tool; the idea is what matters.

Who owns a production bug in code nobody on the team fully read?

The same people who would own it if a human had written it: the engineer who approved the merge and the team that owns the service. The agent is a tool. Its maker's documentation says so in plain terms, and so do the open source security guides written for teams adopting these tools. We cover the accountability question in more depth in who is accountable for AI generated code you submit.

What changes is how you make that ownership real when volume is high.

• The approver owns the decision, not every line. For tier 1 and 2, the approver is accountable for checking that the evidence was present and adequate. For tier 3, the approver is accountable for understanding the code.
• The team owns the service. On call rotations should include the code agents wrote. If nobody on the rotation can debug a module, that is a signal to rewrite or document it, not to exclude it.
• Incidents review the process. Ask which tier the code was in, which evidence was missing and which check would have caught it, then add the check. Track modules that shipped on evidence alone, and review them when they take on a more critical role.

For regulated teams, write the policy down. The SSDF expects documented review of code and third party components, and NIST's SP 800-218A (July 2024) extends it with practices for AI model development. That profile targets model developers rather than teams using coding agents, but the direction is clear: AI in the pipeline sits inside secure development practice, not outside it.

Step by step: a merge gate for agent written pull requests

Here is the whole policy as a sequence a team can set up in an afternoon.

1. Write the tier map and commit it with a CODEOWNERS file that maps tier 3 paths to named reviewers.

2. Turn on branch protection or rulesets. Require pull requests, code owner review and passing status checks, and disallow force pushes to main.

3. Commit the instruction file and the pull request template described above, and set agent permissions per repository.

4. Add CI checks. Tests, a dependency diff check, a software composition scan with a gate, and static analysis. For example, with the Safeguard CLI in a CI job:

   # Scan the checked out repository; results are written as JSON
   safeguard scan --dir .
   

   and, if you build images in the same pipeline:

   safeguard scan --image registry.example.com/app:${GIT_SHA}
   

   Evaluate the results against a gate so a new high or critical finding in a runtime dependency blocks the merge unless someone records an exception.

5. Add the "tests fail on main" check. Even a simple version catches empty tests.

6. Add a second model review as a first pass. Run it on every agent labelled pull request and post its findings as comments for the human reviewer.

7. Measure. Track how many agent pull requests per tier were merged, reverted, or linked to incidents. Adjust the tiers with data.

For a broader checklist on reviewing these pull requests, see reviewing AI generated code in pull requests and review AI generated code before merge.

Common mistakes

Treating "tests pass" as "understood". Tests prove the behaviour you tested. On tier 3 code, the bug is usually in behaviour nobody tested. Require negative tests and human understanding there.

Reviewing the code diff and skimming the lockfile. The lockfile is often the largest and most consequential part of an agent's pull request. A one line manifest change can bring a new transitive tree with known vulnerabilities.

Letting the agent re-resolve the whole dependency tree. A fresh install that rewrites the lockfile mixes intended changes with incidental upgrades. Use npm ci, hash checked pip installs or go mod verify in CI, and ask the agent to change only what it needs.

Accepting a new dependency the standard library already covers. Every new package is new code you did not read, from people you have not vetted. Ask why it was needed.

Mixing tiers in one pull request. A large change that touches UI and authorisation will be reviewed at the UI level. Split it.

Letting two models agree and calling it review. A reviewer model is a filter, not an approver. Someone with context still signs off.

Giving the agent broad permissions on critical repositories. Network access, unrestricted commands and the ability to edit CI workflows without asking are fine in a sandbox and risky on a repository that deploys to production.

Waiting for a maintainer when the problem is in a dependency. If the agent picked a vulnerable version and no fixed release exists yet, there are ways to close the gap now. See the next section.

Fix it without waiting for upstream

When the risk in an agent's pull request is in a dependency it chose, you do not need to block the feature or wait on a maintainer's release schedule. The options below apply to the dependency half of agent written code, which is often the larger half.

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

FAQ

Is it safe to merge AI generated code I haven't read?

For low impact code with good tests and a clean dependency and vulnerability scan, it can be a reasonable risk decision, as long as a named person accepts it. For code that touches authentication, authorisation, money, cryptography, secrets, data deletion, infrastructure or the build pipeline, no. Someone who can explain every line should approve it.

How do I review AI generated code faster without lowering the bar?

Change what you review. Ask the agent for small, single purpose pull requests and a structured explanation, let a reviewer model and static analysis do a first pass, and spend your attention checking the explanation against the diff on the risky files. Automate the dependency diff so you never read a lockfile by hand.

Can I use AI to review AI generated code?

Yes, as a first pass. Both Anthropic and OpenAI document review features for their coding agents. Neither presents them as a replacement for human review, and a reviewer model cannot know what the change was meant to do or what context lives outside the diff.

How do I stop an AI coding agent from adding vulnerable or fake packages?

Tell it not to in its instruction file, require it to justify every new dependency, pin versions in the lockfile, and run a dependency diff check and a vulnerability scan that can block the merge. Verify that each new package exists under the exact name and publisher you expect before approving.

Do tests prove AI generated code is correct?

They prove it does what the tests check. Make sure the new tests fail without the change, add negative cases for anything on a trust boundary, and remember that security bugs are usually in behaviour nobody wrote a test for.

Who is responsible if AI generated code causes a security incident?

The engineer who approved the merge and the team that owns the service, in the same way as for any other code. The tool's makers place responsibility for reviewing proposed code with the user. Use the incident review to find which check was missing and add it.

Does AI generated code have more vulnerabilities than human written code?

The OpenSSF guide for AI code assistants advises developers to assume AI written code can have bugs or vulnerabilities and to review it as carefully as a colleague's work. Rather than relying on general comparisons, measure your own outcomes by tier: reverts, incidents and vulnerability findings per merged pull request.

What is the most overlooked risk in AI written pull requests?

Dependencies. The agent's own lines get read; the packages it pulled in, and the packages those pulled in, often do not. Review the lockfile diff and scan every pull request that changes a manifest. Our post on AI generated code security risks and controls covers the wider set.

See how Safeguard would help

These demos do not review your business logic for you; that stays a human job. They show the dependency half of this post and the approval model we argue for: an assistant that asks before it acts, and that you can scope to the tools it needs.

The first demo is a real CLI session. Asked to fix a lodash CVE in a repository, the Safeguard CLI reads the finding and the advisory, asks before each command, edits package.json to lodash 4.17.21, runs npm install and npm test with approval, prints a summary and pushes nothing. That is the shape of a reviewable agent change: one dependency, one reason, tests run, and a human approving each step.

Click through it at your own pace, or open it full screen.

The second demo sets up the Safeguard runner on your own machine, approves it with its device code, and then scans from the terminal. Running the scan locally before you open a pull request means the dependency evidence is ready when the reviewer arrives.

Click through it at your own pace, or open it full screen.

The third demo connects Claude to the Safeguard MCP server with OAuth and chooses which tools Claude can use. If your team writes code with an assistant, giving it read access to your dependency findings, and nothing more, lets it check a package before proposing it.

Click through it at your own pace, or open it full screen.

To try it on your own projects, download Safeguard or read the CLI documentation. For more on how the CLI approaches coding with security context, see the Safeguard CLI as a security aware coding assistant, and for the broader shift in review practice, do code review practices need to change when half the code is AI written.

Quick check

Sources

• Hacker News: Ask HN: How to deal with shipping AI-written code you don't understand?
• Hacker News: Ask HN: How to enforce engineers to understand the code they are shipping
• NIST SP 800-218, Secure Software Development Framework version 1.1
• NIST SP 800-218A, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models
• OpenSSF: Security-Focused Guide for AI Code Assistant Instructions
• OWASP GenAI Security Project: LLM09:2025 Misinformation
• Anthropic: Claude Code security
• OpenAI: Codex agent approvals and security
• OpenAI: Codex security
• GitHub Docs: About dependency review
• GitHub: dependency-review-action README
• GitHub Docs: About code owners
• npm Docs: npm ci
• pip documentation: Secure installs
• Safeguard CLI documentation

Last reviewed: 23 October 2026. Agent permission settings and defaults change between versions; check your tool's current documentation before relying on them.