Java security tooling forked into two philosophies almost twenty years apart, and most teams never notice they're only using one of them. FindBugs launched in 2006 doing bytecode analysis; its successor SpotBugs now ships more than 400 built-in bug detectors, and the Find Security Bugs plugin layered on top adds 144 security-specific detectors backed by roughly 800 API signatures covering Spring, Struts, JSF, and OWASP-style flaws like SQL injection, XXE, SSRF, and weak cryptography. Meanwhile Semgrep, built for speed, matches syntax patterns against source before a build even runs, and GitHub's CodeQL compiles Java into a queryable relational database to run real taint-tracking dataflow queries. Each of these tools is free and open source, actively maintained, and genuinely useful — but each catches a different slice of vulnerable code, and OWASP's own Benchmark project exists specifically because no single one of them scores well across every weakness category it tests. This piece compares how they actually work, where each one's blind spots are, and how to combine them into a review process that catches more than any single scanner would alone.
Why does bytecode analysis catch different bugs than source analysis?
Bytecode analysis, the approach SpotBugs inherited from FindBugs, works on compiled .class files rather than raw source, which lets it see things a text-based scanner cannot — resolved types, inlined constants, and the actual method signatures the JVM will execute, not just what the source appears to say. That's also its cost: SpotBugs needs a successful compile before it can run anything, so a codebase with a broken build or unresolved dependencies simply can't be scanned. Find Security Bugs extends this same bytecode model with its 144 security detectors, which means it inherits both the accuracy benefit (fewer false positives from misreading generics or overloaded methods) and the build dependency. Source-pattern tools like Semgrep skip the compile step entirely, matching against the syntax tree of the .java file itself, which is faster and works on code that doesn't build yet — but it can miss issues that only become visible after type resolution, such as a sink that only exists because of an inherited interface.
How much deeper does CodeQL's dataflow model go than pattern matching?
CodeQL goes further than either bytecode or syntax pattern matching by building an actual relational database from a full Java build and running queries that trace data as it flows across method calls, interfaces, and even some framework boundaries — true interprocedural taint tracking, not single-file pattern matching. That's the mechanism that lets a CodeQL query say "user input from an HTTP parameter reaches a Statement.execute() call four methods away" rather than just flagging every execute() call in the codebase. The tradeoff is setup cost: CodeQL requires a working, reproducible build in CI (GitHub's Actions-based codeql-action handles this for supported build systems), and query authoring for custom sinks has a real learning curve. It's free for public repositories and open-source projects, which is exactly the profile OWASP Benchmark and similar public test suites use it against, but private-repo use requires GitHub Advanced Security licensing.
Where does Semgrep fit for teams that need CI feedback in seconds, not minutes?
Semgrep fits where speed and incremental adoption matter more than deep interprocedural tracing: its open-core engine, released under LGPL-2.1, parses source directly and can return findings on a changed-files diff in seconds, which is fast enough to run as a pull request gate rather than a nightly job. Because it never needs a compiled artifact, teams can turn it on for a legacy Java module that doesn't build cleanly in CI yet, or for a monorepo where a full CodeQL build takes far longer than a single PR's changed lines warrant. Its Java rule packs cover common categories — hardcoded credentials, unsafe deserialization, weak cipher usage, path traversal — using the same rule syntax across every supported language, which lowers the cost of a team maintaining custom rules across a polyglot codebase. The honest limitation: without paid Pro rules or custom interprocedural configuration, community-tier Semgrep's dataflow reasoning is shallower than CodeQL's, so it's better understood as a fast first filter than a full replacement for a dataflow engine.
Does PMD belong in a security review pipeline at all?
PMD is worth naming precisely because it's often confused with a security tool when it isn't primarily one — it's a source-based static analyzer focused on code quality: unused variables, overly complex methods, copy-pasted logic, and style violations. PMD does ship a handful of security-adjacent rules (like flagging hardcoded cryptographic keys or insecure random number use in some rulesets), but its core value in a review pipeline is different from SpotBugs or Semgrep's: it catches the maintainability and complexity issues that make a codebase harder to review safely in the first place, since a 300-line method with six nested conditionals is exactly the kind of code where a real injection flaw hides between two people's merge conflicts. Teams that pair PMD with SpotBugs are usually optimizing for two separate goals — code health and vulnerability detection — not trying to cover the same ground twice.
How do you objectively compare these tools instead of trusting vendor claims?
The OWASP Benchmark project answers this directly: it's a deliberately vulnerable Java test suite, built specifically to measure a SAST tool's true-positive and false-positive rate against thousands of known-good and known-bad test cases spanning categories like SQL injection, XSS, path traversal, and weak cryptography. Running SpotBugs with Find Security Bugs, Semgrep, and CodeQL against the same Benchmark corpus produces genuinely different coverage profiles per category, which is the empirical basis for the claim that no single free Java tool wins outright — one tool's strong SQL injection recall doesn't imply equally strong XXE recall. That's also why security teams that rely on a single scanner's clean report are making an unverified assumption about coverage rather than a measured one; running a known test corpus like Benchmark against your own tool combination, even a subset of it, turns "we run static analysis" into "we know what our static analysis actually catches."
What does layering these tools look like in a real CI pipeline?
A defensible pipeline runs pattern-based and bytecode/dataflow-based tools in different stages rather than picking one: Semgrep on every pull request as a fast pre-build gate for high-confidence rules (hardcoded secrets, obviously unsafe deserialization calls), SpotBugs with Find Security Bugs on every merge to a protected branch once the build succeeds, and CodeQL on a scheduled or nightly job where its deeper dataflow queries can run against the full compiled codebase without blocking developer velocity. Gating strictly on high-confidence, low-false-positive rules first — rather than every rule every tool ships — is what prevents alert fatigue from burying real findings under noise, a documented failure mode in security tool adoption generally. None of this replaces dependency and SBOM scanning: these are all code-level review tools, and a perfectly clean SpotBugs and CodeQL report says nothing about a vulnerable version of Jackson or Log4j sitting in your pom.xml — that's a separate, necessary layer of software composition analysis running alongside, not instead of, secure code review.