Knockout questions are pass or fail items placed at the top of an RFP so unsuitable vendors drop out before anyone writes a long answer. Good ones are factual, checkable and tied to a hard constraint in your environment. Keep them to ten or fewer and score nothing above them.
What makes a good knockout question
- It has a yes, no or short factual answer.
- A wrong answer means you truly could not use the product.
- You can check it from documentation or a quick call.
- It does not describe a feature you merely prefer.
If you would still consider a vendor that answers no, it is not a knockout. Move it into scored questions.
Ten knockout questions to adapt
| # | Question | Why it is a gate |
|---|---|---|
| 1 | Which languages and package ecosystems are supported today, listed by name? | A gap here means blind spots in your own code |
| 2 | Is the product available as SaaS, self-hosted, or both? | Your policy may allow only one |
| 3 | In which regions is customer data stored and processed? | Residency commitments |
| 4 | Does it integrate with our source control and CI system? | No integration, no adoption |
| 5 | Can we export all findings and inventory data in an open format? | Avoids lock-in |
| 6 | Does it support single sign-on and role based access? | Access control baseline |
| 7 | Does the product send our source code outside our environment? | Data handling constraint |
| 8 | Which capabilities in your response are generally available today, and which are early access or roadmap? | Separates real from planned |
| 9 | Will you provide security documentation under NDA before contract? | Your review depends on it |
| 10 | Will you commit to a named support contact and response targets in writing? | Operations need it |
Question 8 deserves special attention. Ask vendors to label every capability by maturity. It forces honesty and gives you a clean basis for later scoring.
How to word them
Avoid adjectives. "Is the platform secure?" gets a yes from everyone. "Does the product send source code to your cloud, and if so, which components and for how long is it retained?" gets a useful answer.
Ask for a one line answer first and optional detail second. This keeps knockout screening fast.
Handling a near miss
Sometimes a vendor answers "not today, but it ships next quarter." Decide in advance how you will treat roadmap items. A common rule is that roadmap does not count toward a must, though it may count toward a should if there is a dated commitment you can put in the contract.
You may also waive a knockout when a vendor's workaround is credible and tested. Record the waiver, who approved it and why, so it is not a quiet exception.
A worked example: screening six vendors in a day
Suppose a team sends the ten knockout questions to six vendors, with a one page response form and a reply date. Here is a realistic readout.
| Vendor | Q1 ecosystems | Q2 deployment | Q3 region | Q5 export | Q8 status labels | Result |
|---|---|---|---|---|---|---|
| A | Pass | Pass | Pass | Pass | Pass | Advance |
| B | Fail (missing build tool) | Pass | Pass | Pass | Pass | Out |
| C | Pass | Fail (SaaS only) | Pass | Pass | Pass | Out |
| D | Pass | Pass | Unknown | Pass | Pass | Follow up |
| E | Pass | Pass | Pass | Fail (findings only) | Partial | Follow up |
| F | No reply | Out |
Two vendors advance cleanly, two need a follow-up question, and two are gone. The whole screen took a few hours of reading, and nobody had to sit through a demo.
Handling unknowns and partial answers
An "unknown" or a partial answer is not a fail. Send one written follow-up with a deadline. If the answer does not arrive or stays vague, treat it as a fail and note why. Vendors that cannot answer a factual question about their own product in a few days will probably be slow later too.
Writing the response form
Keep it simple:
- One row per question.
- A column for Yes, No or Partial.
- A column for one sentence of explanation, with a word limit.
- A column for a documentation link.
- A line at the bottom for the name and role of the person who confirms the answers are accurate.
Ask someone senior at the vendor to confirm the answers. It raises the care taken, and gives you a name if an answer later proves wrong.
Common mistakes with knockouts
- Using them as scored questions in disguise. If a no is acceptable, it is not a knockout.
- Knockouts that only one vendor can pass. That suggests you copied the vendor's feature list. Rewrite in terms of your environment.
- Skipping the written record. Keep the form. Six months later, you may need to explain a decision.
- Rejecting on a misunderstanding. Read answers twice and allow one clarification.
A short note on maturity labels
Question 8 on maturity deserves repetition because it prevents the most expensive misunderstanding. A roadmap item answered "yes" can lead to a real gap in month three. Define the labels in the RFP: generally available means in production use by customers today; early access means available to a limited group; roadmap means planned but not delivered. Then hold vendors to them.
Where Safeguard fits
When you put question 8 to Safeguard, the GA list includes zero-day discovery, reachability analysis, SBOM and dependency scanning, policies and gates, and autonomous remediation with fix PRs that is off by default. Check each one against your own repos. The free scan at https://gold.safeguard.sh/scan is a quick starting point.
Try it yourself
Click through it at your own pace, or open it full screen. Check which ecosystems and components the inventory covers, and use that as a model for how to phrase your own ecosystem knockout question.
Quick check
How many knockout questions are too many?
More than about ten. Beyond that, you are scoring in disguise and will drop vendors you might have wanted.
Can a knockout question be waived?
Yes, if a tested workaround exists. Record the waiver and the approver.
What do we do with a vendor that does not reply?
Send one reminder with a firm date. If there is still no reply, mark it out and record the reason. A vendor that ignores a written screening question is unlikely to improve after signature.
Last reviewed: 19 October 2026.
