Ask ten things: what is on by default, who approves each change, what the tool can touch, how big a change can get, how tests gate it, how you stop it, how you undo it, what is logged, who can change the settings, and what the model sees. The answers matter more than a feature list.
The questions, with what to listen for
| # | Question | A good answer sounds like |
|---|---|---|
| 1 | What is the default state? | Off, or an explicit manual mode, until we choose |
| 2 | Who approves a change before it reaches a protected branch? | A person on our team, enforced by our repository rules |
| 3 | What access does the tool have to our repositories? | Narrow scopes, revocable, no admin rights |
| 4 | Is there a cap on change size and volume? | Yes, configurable per category |
| 5 | How do our tests gate its changes? | It uses our required checks and cannot skip them |
| 6 | How do we stop it right now? | One switch, documented, tested |
| 7 | How do we undo a merged change? | Ordinary revert, no special tooling |
| 8 | What is recorded about each action? | Who, what, when, inputs, and check results |
| 9 | Who can change modes and scopes? | Named roles, and the change is logged |
| 10 | What data does the model receive? | A written answer we can give our privacy team |
Why these ten
Each maps to a known failure of systems that act on a model's output. OWASP's excessive agency entry lists excessive functionality, excessive permissions and excessive autonomy as root causes, and recommends least privilege, enforcing authorization downstream rather than trusting the model, and human approval for high-impact actions. Questions 2, 3 and 4 follow from that directly.
Questions 5 to 7 are about containing damage. A change that passes weak tests can still break production, so you want to know that your own checks are the gate and that undoing the change is routine. The related post on what green CI does not prove about a fix PR covers the gap.
Questions 8 and 9 are about accountability. OWASP is clear that logging and monitoring do not prevent excessive agency, but they limit the damage and make review possible. If nobody can say who switched a setting, you cannot reconstruct an incident.
Question 10 is the one most often skipped. Ask what source code, dependency data or advisory text leaves your environment, where it is processed, and whether it is retained. Get the answer in writing.
How to use the list
- Send the questions before a demo, not after.
- Ask for evidence for each answer, such as a settings screen or a documentation page, not a verbal assurance.
- Test two of them in a trial: stop the tool, then revert a change.
- Score each as clear, partial or unclear, and treat unclear as a risk to resolve.
A vendor who answers every question with enthusiasm and no specifics has told you something. So has one who says a question is not relevant.
Questions that sound good but tell you little
- "How accurate are your fixes?" Accuracy claims depend on test sets you cannot inspect. Ask how failures are caught instead.
- "Is it AI powered?" Almost everything is. Ask what it can do with no human present.
- "Do you integrate with our tools?" Ask what permissions that integration needs.
Safeguard offers autonomous remediation with fix PRs. They are off by default and every category is set to Manual, which is its answer to question 1. Check the rest of your list against any product, including ours.
Try it yourself
Click through it at your own pace, or open it full screen. Watch how a change moves from finding to a gated decision.
Quick check
Which single question matters most?
The default state and who approves changes. If those are right, a mistake in the other nine is easier to contain.
Should we ask these in a security questionnaire?
Yes, but also test the stop and revert paths in a trial, since written answers can be more optimistic than behavior.
Sources
Last reviewed: 19 October 2026.
