Pen Test Retest Evidence for SOC 2 and Customer Questionnaires

On 10 June 2026 CISA issued Binding Operational Directive 26-04, "Prioritizing Security Updates Based on Risk." It replaces BOD 19-02 and BOD 22-01 and ties remediation deadlines to how exposed an asset is, whether the flaw is on the Known Exploited Vulnerabilities list, whether exploitation is automatable and what the technical impact is. The clock starts when CISA adds the CVE or the agency finds it on an asset, whichever comes first.

You are not a federal agency, but the habit is worth copying: decide your deadlines by risk, start the clock at discovery, and keep dated proof. That proof is what turns a penetration test from a scary document into sales evidence.

What's affected

Any company that has received a penetration test report and now has findings to close. Two audiences will ask what happened next.

Your SOC 2 auditor. Neither a penetration test nor a vulnerability scan is strictly required by the Security criteria. Linford & Company, a CPA firm, wrote in a 2021 article that the simple answer on both is no. It points to two points of focus: CC7.1 includes conducting vulnerability scans on a periodic basis and taking action to remediate identified deficiencies on a timely basis, and CC4.1 lists ongoing and separate evaluations, including penetration testing. Points of focus are illustrative, and your auditor decides what they need. If you describe penetration testing in your system description, expect to be asked for the report and for proof you acted on it.

Your customers. Reviewers ask "have all critical and high findings been remediated and verified?" A report alone cannot answer that. A report plus a retest can.

The PCI Security Standards Council's guidance describes the loop: remediate exploitable vulnerabilities within a reasonable period, then have the tester retest to validate that the new controls mitigate the original risk. It also warns that remediation dragging on for a long time may need a new engagement, because the environment has changed.

How to check if you're exposed

Open your latest report and build this table for every finding:

ID    Severity  Found       Ticket    Fixed in        Retest result
F-01  High      2026-09-02  ENG-412   PR 3318, 09-14  Verified fixed 2026-09-29
F-02  Medium    2026-09-02  ENG-413   PR 3341, 10-01  Pending retest
F-03  Low       2026-09-02  none      none            No decision recorded

Any row with a blank column is an evidence gap. The commonest are findings with no ticket, fixes with no link to the finding, and "fixed" with no independent verification.

To pull fix evidence quickly:

git log --since=2026-09-01 --grep "ENG-41" --pretty='%h %ad %s' --date=short

Commit messages that reference ticket IDs make the table nearly free to build.

What to do now

1. Triage every finding within days. For each one record severity, owner and a target date. Write your own deadlines by risk, for example critical in days, high in weeks, medium in a quarter, and apply them consistently. Those numbers are an illustration; your policy sets yours.
2. Link everything. Finding to ticket, ticket to pull request, pull request to deploy date. A reviewer should be able to follow the chain in one pass.
3. Retest, do not self certify. Ask the testing firm for a retest of high and critical findings. The PCI guidance suggests a retest report with an executive summary, date of original test, date of retest, original findings and results of the retest.
4. Record exceptions properly. If you decide not to fix something, write the reason, the compensating control, the owner and a review date, and have an executive sign it.
5. Assemble the evidence pack in one folder: statement of work, scope, report, finding tracker, pull request links, retest report, exceptions with sign off.
6. Prepare two versions of the story. A one page summary for customers, and the full pack for your auditor.

A questionnaire answer that works:

An independent penetration test of our web application and API was
completed on 2 September 2026. All critical and high findings were
remediated and verified by the testing firm's retest on 29 September
2026. Medium and low findings are tracked with target dates. A summary
letter is available under NDA.

That is an example. Replace the dates and counts with your own and never state a result you cannot evidence.

How to avoid this trap

• Do not let findings live only in a PDF. Move them into your ticket system on day one so each has an owner.
• Do not close findings on a developer's say so. Verification by the tester, or at least a documented reproduction of the original exploit failing, is far more convincing to a reviewer.
• Do not wait until audit week. Evidence recorded as you work is far easier to assemble than evidence reconstructed from memory.
• Do not promise a deadline in a questionnaire that your policy does not set.

Why it matters

A penetration test creates a dated record that you had problems. If you cannot show the same record for the fix, the report becomes a liability. A reviewer who sees ten findings and no closure assumes ten open risks.

With a retest, the story reverses. The same report shows that an outside party looked, found issues, and later confirmed they were fixed. That is the kind of evidence that makes a security review easier.

Try it yourself

Build the finding tracker for your last report in a spreadsheet using the columns above. Time yourself. If it takes more than an hour, your process is not leaving enough trail, and that is the thing to fix before the next test.

Click through it at your own pace, or open it full screen.

Quick check

Sources

• CISA BOD 26-04: Prioritizing Security Updates Based on Risk (10 June 2026)
• Linford & Company: Vulnerability Assessment vs Penetration Testing for SOC 2 Audits (22 June 2021)
• PCI Security Standards Council, Information Supplement: Penetration Testing Guidance v1.1 (September 2017)
• AICPA Trust Services Criteria