To check an AI assistant's claim that a fix works, do not rely on its summary. Read the actual change, run the tests yourself, scan again, and confirm the problem is gone and nothing else moved. A fix is proven by evidence you can see, not by the assistant saying "done."
TL;DR: Treat "I fixed it" as a claim, like a coworker saying the same thing. Six checks turn it into evidence: read the diff, run the tests, rescan, confirm the new version, look for side effects, and record what you checked. It takes about fifteen minutes for a small fix.
Definition: diff. A diff is a side-by-side list of exactly what changed between the old version of your code and the new one. Lines removed are marked one way, lines added another.
Why can an AI assistant be wrong about its own fix?
An AI assistant writes a summary of what it did. That summary is also generated text. It can describe the intended change, not the real one.
Several things can go wrong at once:
- The assistant may fix a different problem than the one you asked about.
- It may change more files than you expected.
- It may make the warning disappear without removing the cause, for example by turning off a check.
- It may say the tests pass when it ran only some of them, or none.
None of this means the assistant is dishonest. It means the words are not the work. The work is in the files.
Our post on the red flags in an AI fix diff lists specific things to look for. This post gives you the whole routine.
What are the six checks?
Use these in order. Each one is cheap, and together they catch most problems.
- Read the diff. Open the list of changes. Ask: is every changed file one I expected? Are there edits that have nothing to do with the problem? For a library upgrade, the usual change is a version number in one or two files.
- Run the tests yourself. Do not rely on the assistant's report. Run the project's test command on your own machine or in your build system, and read the result. If the tests were changed in the same diff, read those changes closely. Our post on whether you can trust tests an AI coding agent wrote explains why.
- Scan again. Run the same scan that found the problem. The old finding should be gone, and no new serious finding should have appeared.
- Confirm the version. If the fix was "upgrade this library," check that the new version is really in use, not just written in a file. Some projects keep a second record of exact versions, called a lockfile, and it needs to match.
- Look for side effects. Did the upgrade skip across a major version? A major version jump (for example from 3.x to 4.x) can change how the library behaves. Run the app and try the main paths by hand.
- Write down what you checked. A two line note in the pull request is enough: what you ran, what you saw, the date.
How do the checks compare in effort and value?
| Check | What it proves | Time for a small fix | What it misses |
|---|---|---|---|
| Read the diff | The change is what you think it is | 3 to 5 minutes | Behavior you cannot see in text |
| Run tests | Existing behavior still works | 1 to 10 minutes | Cases the tests do not cover |
| Rescan | The known problem is gone | 1 to 5 minutes | Problems not in the database yet |
| Confirm version | The fix is truly installed | 2 minutes | Nothing else changed |
| Try the app | Real use still works | 5 minutes | Rare paths |
| Write a note | Others can see your evidence | 2 minutes | Nothing, but it helps audits |
The times are rough guides for a small change. A large upgrade takes longer.
What does a worked example look like?
All of the numbers below are invented for illustration.
An assistant reports: "I upgraded the date library from 2.8.0 to 2.9.4 to fix a known flaw. All tests pass."
You run the six checks.
- Diff. It shows two files changed: the dependency file and the lockfile. That is expected. Good.
- Tests. You run the suite. 212 tests ran, 212 passed. The assistant's claim holds. You note that no tests were edited.
- Rescan. The earlier scan listed 1 high finding on this library. The new scan lists 0 for that library. It also lists 0 new findings.
- Version. The lockfile shows 2.9.4. The app starts and prints 2.9.4 in its version output. Good.
- Side effects. 2.8.0 to 2.9.4 is a minor update, not a major one. You open the page that shows dates and it renders correctly.
- Note. You write: "Checked diff (2 files), tests 212 of 212, rescan clean for this library, version 2.9.4 confirmed. 19 Oct."
Total time: about twelve minutes. The claim was true. You now know it, rather than hoping it.
Now imagine step 1 had shown 14 changed files, including a settings file that turned off a check. You would stop and ask the assistant to explain, or redo the fix with narrower instructions. That is the point of looking.
How do you ask for a smaller, more checkable fix?
You can make verification easier by shaping the request. Try these prompts, adapted to your project.
- "Change only the version of this one library. Do not edit anything else."
- "Show me the diff before you apply it."
- "Do not change or delete any tests."
- "After the change, list the exact commands you ran and what each printed."
Small, narrow requests lead to small diffs. Small diffs are easy to read. Our post on stating requirements first and then verifying goes deeper on this habit.
What if the scan still shows the problem?
Do not accept "it is a false alarm" from the assistant without evidence. Check three things.
First, confirm the scan ran on the new code, not on a cached copy. Second, check that the fixed version number is at or above the version listed as the fix. A good place to learn what a finding says is our guide on how to read a CVE. Third, ask whether the flaw is reachable in your code. A tool that does reachability analysis can show whether your code ever calls the risky part. If it does not, the finding may be lower priority, but that should be a decision a person records.
Where does Safeguard fit in?
Safeguard helps with the scan and the preview steps. Its command line tool scans a folder, repository, container image or bill of materials on your own machine. Only the scan results go to the platform, not your source code. Its safeguard fix command updates vulnerable packages to safe versions, and per the CLI page, --dry-run previews the change and --create-pr opens the pull request. The CLI can also write code when you need a change beyond a version bump, and you review the result in the same way.
On the platform, reachability analysis shows which findings your code can actually trigger. Autonomous fixes with pull requests are available but off by default, with every category set to Manual, so a person decides when to turn them on. You can see a longer walk through in our post on previewing dependency fixes with the Safeguard CLI.
Run the free scan at gold.safeguard.sh/scan to get a before and after picture. The installers are listed on the download page.
Try it yourself
Open the demo below and follow a vulnerability from the finding to the fix. After each step, ask yourself which of the six checks would apply.
Click through it at your own pace, or open it full screen.
Quick check
Is it enough that the tests pass?
Not on its own. Passing tests show that the things tested still work. They do not show that the vulnerability is gone. Rescan to confirm the finding disappeared.
What is the fastest check if I only have two minutes?
Read the list of changed files. If the assistant changed files you did not expect, you have learned the most important thing quickly.
Should I ever let an AI delete or edit tests as part of a fix?
Only with a clear reason that a person approves. Edited or deleted tests can make a broken fix look healthy, so read those changes line by line.
Do I need to repeat all six checks for every tiny fix?
For a small version bump, the routine takes about a quarter of an hour and gets quicker with practice. For risky areas such as sign in or payments, keep every step.
Sources
Last reviewed: 19 October 2026.
