Security teams that run Trivy across dozens of CI pipelines have started noticing the same failure signature: scans that hang for minutes, then time out with a lock error, or a vulnerability database that quietly balloons from roughly 500-600MB on first pull to several gigabytes after a few months of daily updates. The common thread is a piece of infrastructure most engineers never chose to run: bbolt, the embedded key-value store built by the etcd project (go.etcd.io/bbolt), which Trivy uses as the on-disk format for its local trivy.db cache. Because bbolt allows exactly one writer at a time and never shrinks its file on disk, concurrent scanners on shared runners and long-lived cache volumes eventually hit the same wall: lock contention, disk bloat, and scans that fail for reasons that have nothing to do with the code being scanned. Below, we walk through why this happens, how it shows up in practice, and what it means for teams that depend on scan results being available on demand.
What does "Trivy etcd exhaustion" actually mean?
It means Trivy's local vulnerability database is exhausting the storage engine that etcd itself is built on, not that a Kubernetes etcd cluster is involved. Trivy does not run etcd as a service. Instead, since it moved off its original BoltDB-based cache format, Trivy stores its offline vulnerability feed in a file called trivy.db, formatted using bbolt, the fork of BoltDB that the etcd maintainers took over and now ship as go.etcd.io/bbolt. That library is the same durable, single-file, memory-mapped storage engine that backs etcd's own key-value store. When practitioners say Trivy has an "etcd exhaustion" problem, they're describing what happens when that embedded engine is pushed past the usage pattern it was designed for: single-process, low-concurrency reads with occasional writes, not dozens of parallel CI containers hammering the same cache path every few minutes.
Why does Trivy embed etcd's storage engine in the first place?
Trivy embeds bbolt because it needed a dependency-free, single-file database it could ship inside a static Go binary without requiring a running database server. That is a reasonable engineering trade-off for a CLI tool meant to run anywhere, from a laptop to an ephemeral GitHub Actions runner, and it is why bbolt (and its predecessor BoltDB) is one of the most widely embedded storage engines in the Go ecosystem, used by projects like Consul and, of course, etcd. The trade-off is that bbolt uses copy-on-write B+trees backed by an mmap'd file, and it holds a single OS-level file lock for the entire database during writes. On a developer's laptop scanning one image at a time, that design is invisible. On a CI fleet running Trivy across dozens of concurrent jobs against a shared TRIVY_CACHE_DIR, or an NFS-backed volume mounted by multiple runners, that same design becomes the bottleneck.
How does bbolt's single-writer lock break concurrent scans in CI/CD?
It breaks them by forcing every parallel Trivy process that touches the same cache file to wait for an exclusive lock, and by default that wait has no timeout, so jobs stall instead of failing fast. Bbolt's Open() call accepts an optional Timeout value; when it is left unset, a process trying to acquire the write lock on trivy.db will block indefinitely until the lock is released. In a typical setup, a pipeline stage first runs trivy image --download-db-only to refresh the database, then several downstream jobs run scans against the same cache path in parallel to save time. If the refresh job and a scan job overlap, or if two scan stages start within the same window, one of them sits waiting on the lock rather than returning a clear error. Teams that have hit this describe watching a scan step run for 10-15 minutes past its normal 30-60 second runtime before the CI platform's own job timeout kills it, at which point the pipeline reports a scan failure that has nothing to do with any vulnerability in the image.
Why does the trivy.db cache file grow without bound?
It grows because bbolt reclaims freed pages internally but does not return that space to the operating system, so a database file's size is a high-water mark, not a reflection of its current contents. Every time Trivy runs its default update job, roughly every six hours by design, it writes a fresh set of advisory records into the same trivy.db file and marks the old pages as free for reuse. Bbolt will reuse those freed pages for future writes, but it will not ftruncate the file back down, so the file only ever grows or holds steady, it does not shrink on its own. In long-running environments where ~/.cache/trivy is mounted as a persistent volume, such as a self-hosted GitLab runner or a Jenkins agent that is never wiped, operators have reported the cache directory climbing from an initial download in the 500-600MB range to multiple gigabytes within a few months. On memory-constrained CI containers, typically the 2-4GB limits common on shared runner tiers, mapping a multi-gigabyte file into memory alongside the rest of the scan process's working set is enough to push the container into an OOM kill, which again surfaces to the team as a flaky, unexplained scan failure rather than a capacity problem.
What else compounds Trivy's scan reliability problems?
The bbolt cache issue rarely shows up alone, because it sits on top of a database-distribution model that already has a documented history of rate-limit failures. Trivy originally pulled its vulnerability database from GitHub Releases, and after that mechanism repeatedly hit GitHub's API rate limits under heavy CI usage, the Aqua team moved distribution to OCI registries, publishing trivy-db and trivy-java-db as artifacts on ghcr.io/aquasecurity with a fallback mirror on Amazon ECR Public. That change reduced but did not eliminate the problem: fleets of ephemeral, unauthenticated CI runners pulling from ghcr.io at scale can still trip anonymous-pull throttling, returning TOOMANYREQUESTS-style errors during the exact --download-db-only step that competes for the bbolt write lock described above. Layer in Trivy's in-memory approach to unpacking and analyzing large container images and monorepo filesystems, which can spike RSS well past 1-2GB on images with dense package manifests, and you get a scanner whose failure modes are dominated by infrastructure plumbing, lock files, registry throttling, and memory ceilings, rather than by the vulnerability detection logic teams actually care about.
What's the real cost to security teams?
The real cost is that "scan failed" becomes a routine, ignorable event instead of a signal, and that erodes the entire point of running a scanner in the pipeline. When a lock timeout or an OOM kill produces the same red X in CI as an actual policy violation, engineers learn to re-run the job and move on rather than investigate, and pipelines start adding retry loops around the scan step just to get a green build. That habit is exactly how a real finding gets missed: if a scan silently fails and a retry happens to succeed against a partially-refreshed or corrupted trivy.db, teams can ship an image believing it was checked when the check never actually completed against current advisory data. Compliance programs that rely on documented, gate-blocking scan evidence for SOC 2 or supply-chain attestations inherit that same gap, because a flaky scanner produces an inconsistent audit trail, not a missing one, which is harder to catch after the fact.
How Safeguard Helps
Safeguard was built around the assumption that a security gate has to be as reliable as the pipeline it sits in, so we do not ask every CI runner to manage its own embedded, single-writer database file. Vulnerability and SBOM data is served from a centrally managed, versioned data plane rather than a local bbolt cache that each job downloads, locks, and grows independently, which removes the concurrent-writer contention and unbounded disk growth that drive Trivy's etcd-derived reliability issues in the first place. Scan requests from parallel jobs read from that shared, always-current source instead of racing each other for a file lock, so a burst of concurrent pipeline runs does not turn into a queue of stalled jobs waiting on a timeout. Database freshness is handled as a managed service update, not a per-runner cron job pulling from a rate-limited public registry, which closes the other half of the reliability gap. And because every scan result is logged as a discrete, attributable event rather than a pass/fail exit code from a local binary, teams get the audit trail that compliance frameworks expect: a scan either ran against current data and produced a result, or it is visibly and explicitly marked as not having run, with nothing in between. For teams that have grown used to treating a red scan step as noise, that distinction, a gate that fails loudly when it should and stays quiet when it shouldn't, is the difference between a scanner that reports risk and one that manages it.