Bandit, the PyCQA-maintained scanner that started life as an OpenStack Security project, ships a broad set of built-in checks that walk Python's ast module looking for known-dangerous syntax patterns — B307 for eval(), B602 for subprocess.Popen(shell=True), B301 for unsafe pickle loads. It runs in milliseconds on a 50,000-line codebase because it never executes anything and never traces where a value came from; it just pattern-matches syntax nodes. That design choice is also its ceiling. A separate class of tools — GitHub's CodeQL and Semgrep's taint-mode rules — instead model data flow from sources like request.args to sinks like os.system, catching indirect paths AST tools can't see. A third class, the type checkers (mypy, Pyright, Pyre), lean on typeshed, the community stub repository at github.com/python/typeshed, to reason about code without running it at all. Each of these three approaches — AST pattern-matching, taint tracking, and type-stub-driven inference — solves a real problem the others don't, and each one has a specific, well-documented blind spot that lets whole categories of bugs through. This post walks through all three and where they fail.
What does an AST-based scanner actually see?
An AST-based scanner sees syntax, not behavior. Bandit parses your source into Python's abstract syntax tree and walks it looking for call nodes that match a known-dangerous shape: a call to eval, a subprocess invocation with shell=True (B602) or without it (B603), a hardcoded string assigned to a variable named password (B105). Because it works one function at a time on syntax alone, it has no concept of where an argument's value originates — it will flag eval(user_config["trusted_key"]) exactly as loudly as eval(request.args["cmd"]), because both are syntactically "a call to eval." That produces the two failure modes every Bandit user learns to live with: noisy false positives on genuinely safe uses, and silent misses when a dangerous call is reached through an alias, a wrapper function, or string concatenation split across variables — none of which change the AST shape Bandit is matching against.
Why does taint tracking catch what AST tools miss, and where does it break?
Taint tracking catches what AST tools miss because it follows a value's journey through the program instead of matching syntax where it lands. CodeQL and Semgrep's taint-mode rules define sources (request.args, input(), sys.argv) and sinks (os.system, cursor.execute, pickle.loads), then build a data-flow graph connecting them — flagging os.system() only when its argument actually traces back to untrusted input, not on every call site. That precision comes at a cost: Python's dynamic features routinely break the flow graph the analysis depends on. getattr/setattr resolve attribute names at runtime, so a taint engine often can't tell which method a dynamically dispatched call actually invokes. Monkey-patching reassigns a function after the module loads. Decorators wrap and replace functions, sometimes changing their signatures entirely. And exec/eval building code from strings at runtime is, definitionally, code the analyzer never sees as code at all — it's just a string literal until the interpreter runs it. Both CodeQL and Semgrep document this: Semgrep's taint mode doesn't build a full call graph, so some inter-procedural paths are missed outright, while CodeQL's more exhaustive whole-program model still can't reason about a name it genuinely cannot resolve statically.
Why do type checkers depend on stub coverage they don't control?
Type checkers depend on stub coverage they don't control because most of the Python ecosystem was never written with type annotations at all, and someone has to backfill them from outside the project. mypy, Pyright, and Pyre all lean on typeshed (github.com/python/typeshed), the community-maintained collection of .pyi stub files covering the standard library plus hundreds of third-party packages distributed as separate types-* PyPI packages — types-requests, types-PyYAML, and so on — released independently of the libraries they describe. When a stub exists and is accurate, the checker can flag a real bug, like passing a str where an Optional[int] is expected, without running a line of code. When no stub exists, the checker falls back to treating the object as implicitly Any, which means it silently stops checking that value's flow entirely — not a warning, just an analysis blind spot. Worse, a stub that's stale relative to the library's current behavior produces false confidence: the checker reports clean, and the underlying code has already diverged from what the stub describes.
What blind spots show up across all three approaches?
Certain patterns defeat all three approaches simultaneously, which is why security teams that rely on a single technique keep getting surprised. Dynamic imports via importlib.import_module() build the module path at runtime, so neither an AST scanner nor a taint engine can know in advance which module — or which of its functions — is actually being invoked. Deserialization sinks reached through generic wrapper functions (a shared load_config() helper that calls pickle.loads() internally, used across a dozen call sites with different trust levels) require the analyzer to model every caller's trust level individually, which most rule sets don't do out of the box. ORM query builders launder taint through method chains — Model.objects.filter(**kwargs) — in ways that break simple source-to-sink tracing unless the tool has a framework-specific model for that exact chain. And multi-file, framework-level flows through Django or Flask request objects only get caught when the tool has been explicitly configured with that framework's sources and sinks; out of the box, a generic taint ruleset has no idea that request.GET is untrusted input at all.
Does running all three techniques together close the gap?
Running AST, taint, and type analysis together closes most of the gap but not all of it, because their blind spots only partially overlap rather than canceling out completely. An AST scanner's noise on safe eval() calls doesn't help a taint engine that's already lost the flow through a decorator; a type checker's Any fallback on an unstubbed third-party package doesn't get fixed by adding a taint rule for a source that library doesn't touch. What layering the three does reliably improve is coverage of the cases where the assumptions differ — a bug an AST tool would flag as noise might be exactly the case a taint engine confirms as reachable, and a value a taint engine loses through getattr might still be one a type checker's stub correctly types as Any and flags for manual review. The honest takeaway for a Python security program isn't "pick the best tool," it's that none of the three techniques was designed to answer the same question, and treating one as a complete substitute for the others is exactly how dynamic-import sinks, wrapper-laundered pickle calls, and stub-less third-party packages end up shipping unreviewed.