Test each control with a harmless canary: a fake secret in a protected file, a forbidden command, a pull request that should be blocked. If the control does not stop the canary, it is not a control. Run the tests when you roll out a setting, after tool updates, and quarterly.
Why test
Vendor documentation lists gaps. GitHub says content exclusion does not apply to symbolic links or remote filesystems, and that support varies by tool and mode. Claude Code documents that file deny rules do not cover every subprocess. Your own configuration adds more ways to be wrong. Only a test tells you whether the control holds in your environment.
Seven tests
| Test | How | Pass looks like |
|---|---|---|
| File exclusion | Put a canary value in a protected file, ask the assistant to summarize it | It cannot read or use the file |
| Agent deny rule | Ask an agent to open the same file | The read is blocked |
| Subprocess path | Ask it to read the file through a script | Blocked by sandbox, or you record the gap |
| Command limits | Ask for a denied command | Refused or prompts every time |
| Secret scanning | Commit a fake token in a test branch | Blocked locally or at push |
| Review gate | Open a pull request touching a code owner path | Cannot merge without owner approval |
| Logging | Perform an action, then look for it in logs | Entry exists with user and time |
Use safe canaries
Use clearly fake values, such as a string that matches a scanner pattern but is not a live credential. Never use a real key. Create canaries in a test repository so a failed test leaks nothing real.
Record results
Keep a short log: date, tool and version, test, result, and who ran it. If a test fails, record the compensating control and the date you will retest. This log is also strong evidence when a customer asks how you govern AI tools.
When to retest
- After any major tool or model update.
- After changes to organization settings.
- When a new mode or feature is turned on.
- Every quarter, even if nothing changed.
Try it yourself
Click through it at your own pace, or open it full screen.
Click through the gate and treat each failing check as the result you would expect from your review gate test.
Quick check
Who should run the tests?
Someone other than the person who configured the control, ideally a developer and a security engineer together.
Is a passing test permanent?
No. Tools change often, so retest after updates.
Safeguard policies and gates provide a testable merge-time control, and a free scan is at gold.safeguard.sh/scan.
Sources
Last reviewed: 19 October 2026.
