OpenAPI driven testing cannot find what the spec does not describe, and it cannot judge business meaning. It will miss undocumented endpoints, many authorization flaws that need two real users, abuse of legitimate flows, and anything that depends on realistic data. It is a map, not a tester's judgment.
What the import actually does
Scanners that accept an OpenAPI file read each operation, then send requests built from it. The ZAP OpenAPI add-on is a typical example. It imports definitions in several OpenAPI and Swagger versions, adds endpoints to the site tree, and uses default values for parameters. Its documentation notes that XML content generation is not supported and that supplying your own parameter values needs a separate add-on.
That is a fair picture of the category: good at breadth, limited by what it is told.
Five blind spots
| Blind spot | Why a spec import misses it |
|---|---|
| Undocumented endpoints | Not in the file, so never requested |
| Object level authorization | Needs valid IDs that belong to a different user |
| Function level authorization | Needs sessions for each role, not one token |
| Business flow abuse | Each call is valid, the sequence or speed is the problem |
| State dependent bugs | Needs earlier calls to create the right data |
OWASP's description of API6 is useful here. It says detection relies on non human patterns, such as moving from adding to a cart to completing a purchase faster than a person could. A request by request tester sees nothing wrong with any single call.
How to close each gap
- Undocumented routes. Combine the import with discovery from logs and code, as in finding undocumented API endpoints.
- Object authorization. Provide two accounts with seeded records and test cross access, following how to test for BOLA with two accounts.
- Function authorization. Provide a session per role and use a per operation matrix.
- Business flows. Add rate and sequence tests written by someone who knows the product.
- State. Write setup scripts that create realistic records before the run.
Where spec driven testing is strong
It is cheap, repeatable and runs in CI. It finds missing authentication on documented routes, verbose errors, open HTTP methods, missing limits and schema violations. Those are real issues, and they cost little to catch on every build.
The mistake is treating a clean result as proof of safety. Treat it as proof that the documented surface passed the automated checks you configured.
Try it yourself
Click through it at your own pace, or open it full screen. Ask which of the findings depend on knowing who owns a record.
Quick check
Is a spec import enough for compliance evidence?
Usually not on its own. Auditors and buyers tend to want to see scope, frequency and how findings were handled, plus testing of authorization.
Should I still import the spec?
Yes. It gives breadth quickly. Just add the human designed tests it cannot generate.
Safeguard's free scan at gold.safeguard.sh/scan checks dependency risk, which a spec import never sees.
Sources
- ZAP OpenAPI Support add-on
- OWASP API6:2023 Unrestricted Access to Sensitive Business Flows
- OWASP API1:2023 Broken Object Level Authorization
Last reviewed: 19 October 2026.
