Aikido Security, ScanRepeat, and a growing wave of "AI pentest" tools now promise same-day penetration test reports generated by large language models probing your app for SQL injection, broken auth, and misconfigured APIs. The pitch is seductive: skip the six-week wait for a human tester, skip the five-figure invoice, and still walk into your SOC 2 Type II audit or PCI DSS assessment with a clean report in hand. For a startup racing to close enterprise deals, that sounds like exactly what compliance needs.
It isn't quite that simple. SOC 2, ISO 27001, HIPAA, and PCI DSS each have specific, sometimes explicit, requirements about what a "penetration test" must include — who performs it, what methodology it follows, and what evidence it produces. Some of those requirements were written assuming a human tester. This post breaks down what each framework actually demands, where AI-only pentesting tools fall short, and where they genuinely help.
Does SOC 2 require a human-performed penetration test?
Not explicitly, but auditors almost always expect one in practice. SOC 2's Trust Services Criteria (CC7.1, CC4.1) require organizations to identify vulnerabilities through "testing" and to monitor the effectiveness of controls, without naming a specific tool or methodology. The AICPA leaves the "how" to the auditor's judgment. In practice, the CPA firms issuing SOC 2 Type II reports — the same ones evaluating evidence for a 6-12 month observation period — have converged on expecting an independent, third-party penetration test at least annually, with a signed report describing scope, methodology (commonly referencing OWASP Testing Guide or PTES), findings, severity ratings, and remediation timelines. An auditor reviewing a report generated entirely by an AI scanner, with no named tester, no manual exploitation attempts, and no attestation of independence, will typically flag it as insufficient evidence for CC7.1 and request a supplemental test. Aikido's AI pentesting output can supplement this evidence trail, but auditors we've seen in 2024-2025 SOC 2 cycles still ask for a human-attested report alongside it.
Does ISO 27001:2022 accept an AI-generated report as evidence for Annex A 8.29?
Only as one input, not the whole answer. Annex A control 8.29 ("Security testing in development and acceptance") requires that security testing processes be defined and implemented, and Clause 9.1 requires organizations to evaluate the effectiveness of those processes through monitoring, measurement, analysis, and evaluation. ISO 27001 doesn't mandate a specific pentest cadence or a human tester by name, which is actually why some vendors market AI pentesting hard into this framework — the standard's language is broad enough to fit. But certification body auditors conducting Stage 2 audits still assess competence of whoever performed the testing under Clause 7.2 ("Competence"), and an AI tool with no named, qualified assessor behind it creates a documentation gap auditors routinely cite as a minor nonconformity. We've seen ISO auditors accept automated DAST/SAST scan output as continuous monitoring evidence, while still requiring a periodic test performed or reviewed by a competent person (CREST, OSCP, or equivalent) for the annual risk treatment cycle.
Can AI pentesting alone satisfy HIPAA's Security Rule risk analysis requirement?
No — HIPAA doesn't require penetration testing at all, which makes this trickier, not easier. The HIPAA Security Rule (45 CFR § 164.308(a)(1)(ii)(A)) requires a documented, ongoing "risk analysis" covering all ePHI systems, but the word "penetration test" appears nowhere in the regulation text. OCR guidance and settlement precedents (e.g., the 2018 Anthem settlement, which cited inadequate risk analysis rather than a missing pentest) show that regulators care about the completeness and accuracy of the risk analysis, not the tool used to generate one data point within it. An AI pentest report can feed into that risk analysis as evidence of technical vulnerability testing, but it cannot replace the broader risk analysis covering administrative safeguards, workforce access reviews, business associate agreements, and physical safeguards. Covered entities that treat an Aikido-style automated scan as their entire HIPAA technical safeguard evidence typically fail their next OCR audit or breach investigation because the risk analysis is incomplete, not because the scan itself was invalid.
Does PCI DSS 4.0 allow automated or AI-driven testing to replace manual penetration testing?
No, and this is the framework where the gap is most explicit. PCI DSS v4.0, which became mandatory for all Requirement 11.4 sub-requirements on March 31, 2025, requires penetration testing that follows "an industry-accepted penetration testing methodology" (11.4.1) and explicitly requires testing to include both automated and manual techniques, performed by a "qualified internal resource or qualified external third party" with organizational independence from the system being tested (11.4.1, testing note). The standard also requires network segmentation testing every six months for segmented cardholder data environments (11.4.5) and testing after any significant infrastructure or application change (11.4.3-11.4.4). Qualified Security Assessors (QSAs) reviewing evidence for a Report on Compliance have been explicit in industry guidance that fully automated scanning tools, AI-driven or not, do not meet the "manual techniques" bar on their own — they're treated as complementary to, not a substitute for, tester-driven exploitation and business-logic testing. A merchant relying solely on an AI pentest tool for its annual PCI 11.4 test risks a QSA marking that requirement as "not in place."
Where do AI pentesting tools actually fit in a compliance program?
They're strongest as continuous, in-between-audit coverage, not as the annual test of record. Human penetration tests are expensive (typically $8,000-$30,000+ per engagement) and infrequent (annually or per PCI's change-triggered cadence), leaving long windows where new vulnerabilities from dependency updates, new endpoints, or config drift go undetected. AI and automated pentesting tools like Aikido's can run continuously or on every deploy, catching regressions between the formal tests that auditors require. The most defensible compliance posture we see in 2025 combines both: automated/AI testing for continuous coverage and fast triage, paired with a named, qualified human tester's report for the specific control evidence (SOC 2 CC7.1, PCI 11.4.1, ISO 8.29) that auditors are trained to expect. Framing AI pentesting as a replacement rather than a complement is the single most common reason we see these findings get flagged during audit fieldwork.
How Safeguard Helps
Safeguard approaches this the same way auditors do: evidence needs a chain of custody, a named methodology, and a human accountable for the findings, with automation doing the continuous heavy lifting underneath. Our platform runs continuous software composition analysis, SAST, secrets scanning, and dependency risk monitoring across your supply chain, generating the kind of always-on vulnerability evidence that satisfies ISO 27001's Clause 9.1 monitoring requirement and feeds SOC 2's continuous monitoring criteria between formal test cycles. For the point-in-time penetration test that SOC 2, ISO 27001, HIPAA risk analyses, and PCI DSS 4.0's Requirement 11.4 all still expect, Safeguard coordinates with qualified, independent human testers and packages the resulting report — scope, methodology, tester credentials, findings, and remediation SLAs — into the same evidence repository your auditor already pulls from. That means no scramble to reconcile an AI-generated PDF with what a QSA or SOC 2 assessor is trained to look for, and no compliance gap when a regulator asks who actually tested the system. If you're evaluating whether your current pentesting approach will hold up under your next audit, our team can walk through your specific framework requirements and evidence gaps before your assessor does.