In 2021, NYU researchers Pearce, Ahmad, Karri, and Dolan-Gavitt generated 1,689 programs with GitHub Copilot across 89 scenarios mapped to MITRE's Top 25 CWEs and found that roughly 40% contained vulnerable code — about 50% for C and 39% for Python ("Asleep at the Keyboard?", arXiv:2108.09293). Two years later, Stanford researchers Perry, Srivastava, Kumar, and Boneh ran a controlled study and found something worse than the raw vulnerability rate: developers who used an AI assistant wrote measurably less secure code and were simultaneously more confident it was safe. Follow-up work through 2023–2024 (arXiv:2310.02059) found the rate had dropped but was still substantial — around 29.5% of AI-generated Python snippets and 24.2% of JavaScript snippets carried security weaknesses. None of this is a reason to stop using AI coding tools; it's a reason to stop assuming static analysis alone can catch what they produce. This post makes the technical case for why AI-generated code specifically needs dynamic application security testing (DAST), and walks through the bug classes — IDOR, SSRF, race conditions, misconfiguration, auth bypass — that live only at runtime and that a source-code scanner is structurally incapable of seeing.
How bad is the vulnerability rate in AI-generated code, really?
The numbers vary by study design but land consistently in the same range: somewhere between a quarter and a half of AI-generated code samples contain a security weakness when checked against a fixed rubric. Pearce et al.'s 2021 Copilot study found ~40% overall across CWE-mapped scenarios, with SQL injection and hardcoded credential patterns among the most common. The 2023–2024 replications, testing newer models, found lower but still material rates — roughly 30% for Python, 24% for JavaScript. The trend line is improving, not solved. And a separate 2024 study of Java security-API usage (arXiv:2404.03823, Mousavi et al.) found that ChatGPT misused security APIs in roughly 70% of generated code instances across the tasks tested — plausible-looking calls whose flaws only surface once the code is actually exercised at runtime, not by reading the source. That's the throughline: these aren't syntax errors a linter catches. They're semantically valid code that behaves insecurely only in a running system.
Why is the Stanford "false sense of security" finding the most dangerous part?
Perry et al.'s central finding wasn't just that AI-assisted developers wrote less secure code — it's that they rated their own code as more secure than developers who wrote it unassisted. That combination is the dangerous one: a team that knows it's shipping risky code will compensate with extra review; a team that's confidently wrong won't. Static analysis tools can reinforce this false confidence if a team treats "SAST passed" as a security sign-off, because SAST is specifically the layer that cannot see the classes of bug this population is prone to introducing. If an LLM generates an API endpoint that checks authentication but not authorization — a very common Copilot-era pattern — a clean SAST report and a developer's inflated confidence combine into exactly the gap an attacker needs.
Why can't SAST catch the bugs LLMs actually tend to introduce?
SAST works by tracing untrusted input from a source to a dangerous sink through the source code's control and data flow — it is fundamentally a static, single-codebase analysis. It cannot observe: the actual HTTP response headers a deployed server sends, whether a session token is checked against the requesting user's own resource (broken object-level authorization, OWASP API Security Top 10's #1 risk since the list's inception), whether a server-side request forgery is actually reachable because the target URL is only assembled at runtime from three separate config values, or whether a race condition exists between two goroutines an LLM wrote independently and never tested concurrently. LLMs are prone to exactly these classes because they generate one function at a time, optimizing for local plausibility — the code compiles and looks idiomatic — without ever executing it against a live database, a real second concurrent request, or a genuine external service boundary.
What is IDOR/BOLA and why do LLMs produce it so often?
Insecure Direct Object Reference — formalized in the OWASP API Security Top 10 as Broken Object-Level Authorization (BOLA) — happens when an endpoint fetches a resource by an ID supplied in the request without verifying the requester actually owns it. LLMs generate this pattern constantly because the "happy path" version — SELECT * FROM orders WHERE id = :id gated only by "is the user logged in" — is the version most represented in training data and tutorials; the ownership check is a business-logic detail that has to be inferred from the specific data model, not copied from a pattern. A SAST scanner sees a parameterized, injection-safe query and reports nothing wrong. Only a dynamic test that authenticates as user A and requests user B's resource ID actually proves the authorization gap exists.
How does SSRF slip past static review in AI-generated integration code?
Server-side request forgery shows up constantly in the "glue code" LLMs are asked to write — webhook handlers, PDF-from-URL generators, image-proxy endpoints, and third-party API wrappers. The vulnerable pattern is an outbound HTTP call whose target host is built, even partially, from user input, without a validated allowlist. Static analysis can flag the naive case ( requests.get(user_url) ) but frequently misses variants where the LLM assembled the URL from config, a redirect-following client, or an internal service discovery call that only resolves to an internal IP at runtime. DAST catches this differently: it sends a live request designed to prove out-of-band or internal reachability and observes the actual network behavior — something no static trace can simulate, because the vulnerability's existence depends on infrastructure that only exists once the app is deployed.
How does Safeguard combine SAST and DAST to close this gap?
Safeguard's application security testing runs first-party SAST and DAST through a shared, tenant-isolated findings model rather than as separate tools you correlate by hand. SAST traces the source-to-sink dataflow in your code and produces a CWE-mapped finding with the exact trace; DAST tests the running, ownership-verified application with safe, non-destructive requests under configurable safety modes (passive, safe_active, authenticated_safe_active, with a locked-down lab_aggressive mode reserved for owned, isolated targets). When a DAST-confirmed runtime issue — an IDOR, an SSRF, a misconfigured header — maps to the specific source-code sink that caused it, the two findings are linked and prioritized as one confirmed, exploitable issue instead of two disconnected alerts. For teams shipping AI-generated code at LLM speed, that correlation is the practical answer to the Perry et al. finding: it replaces a developer's misplaced confidence with an actual runtime observation.