Verizon's 2026 Data Breach Investigations Report says 31% of breaches now begin with the exploitation of a software vulnerability, the top initial access method in the report. The incidents it covers occurred between 1 November 2024 and 31 October 2025. Security reviewers know that pattern, and a line like "Do you perform regular security testing?" is now routine in enterprise questionnaires.
The trouble is that "security testing" is not one activity. A customer who writes "annual penetration test" and a vendor who answers with a scanner export are talking past each other, and the deal stalls for weeks while both sides work that out.
What's affected
Any small software company answering a questionnaire, a security addendum or an RFP. Four terms get used loosely, so here is what each one means in practice.
Vulnerability scan. Automated tools look for known weaknesses and misconfigurations. The PCI Security Standards Council's penetration testing guidance describes a scan as typically a variety of automated tools combined with manual verification of identified issues, taking seconds to minutes per host, with findings ranked by CVSS score.
Penetration test. A person tries to exploit weaknesses to get around your security controls. The same guidance calls it a manual process that may use scanners or other automated tools, runs for days or weeks, and produces a report describing each verified vulnerability and how far it could be exploited.
Red team. A goal-based exercise where testers act like a specific adversary over a longer period, often to see whether your detection and response notice them. It measures your defenders, not your code coverage. AWS treats red, blue and purple team work and covert simulations as separate from ordinary penetration tests: you submit a Simulated Events form at least two weeks before the start date.
Bug bounty. Outside researchers report issues on their own schedule and you pay for valid ones. It needs a published policy, and ideally safe harbor language. The disclose.io project describes safe harbor as policy language that lets people acting in good faith report issues without fear of legal repercussions. A bounty gives you no guaranteed coverage and no dated report to hand to a buyer.
How to check if you're exposed
Open your last questionnaire and highlight every testing word. Then sort each phrase into one of these buckets:
- "Vulnerability assessment", "scan results", "continuous scanning": a scan export or a dashboard screenshot answers it.
- "Penetration test", "pentest report", "third party assessment": a dated report from a tester outside your engineering team.
- "Red team exercise": rare in small vendor reviews. Ask before you assume.
- "Bug bounty" or "responsible disclosure": a public policy page, a security.txt file and a contact that answers.
If a phrase does not fit a bucket, that is the one to ask about.
What to do now
- Copy the exact testing questions into one document, with the customer name and the date received.
- Send the customer's security contact one clarifying email: "Do you need a third party penetration test report, or is a summary letter acceptable? Is a vulnerability scan with remediation dates enough for the scanning question?" Most reviewers answer in a day.
- Decide per question which artifact you can produce today. A scan export you already have beats a penetration test you will commission next month.
- If a real penetration test is required, scope it before you shop for quotes. A one page scope document saves a round of revisions.
- Never relabel scanner output as a penetration test. Reviewers read both kinds of report for a living and the difference is obvious.
How to avoid this trap
- Do not buy a "pen test" because a questionnaire used the word. Confirm the artifact first.
- Keep scanning running between tests so the scan question always has a fresh answer.
- Publish a disclosure policy before you consider a bounty. Without one, a bounty has no rules.
Why it matters
Buyers use these words as shorthand for a level of assurance. A scan says you look for known problems. A penetration test says someone tried to break in and told you what they found. Mixing them up either under delivers, which costs you the deal, or over delivers, which costs you money you did not need to spend.
A short, precise answer helps too. "Independent penetration test completed on this date, retest on this date, summary available under NDA" tells a reviewer more than a page of general assurance.
Try it yourself
Take the last security questionnaire you received and classify every testing question into the four buckets above. Count how many need a person's judgment and how many only need an export.
Click through it at your own pace, or open it full screen.
Quick check
Your customer asks for "regular vulnerability assessments". Is a penetration test report the right answer?
Usually not. A vulnerability assessment is scan driven. Send scan results with dates and remediation status, and offer the penetration test summary as extra context if you have one.
Why is a bug bounty not a substitute for a penetration test?
A bounty depends on outside researchers choosing to look, so coverage is not guaranteed and there is no scoped, dated report. It complements a test; it does not replace one.
