Log flooding: how attackers and bugs overwhelm your log pipeline

A log flood is a surge of events that fills disks, saturates collectors, hides real activity or inflates your bill. Attackers can cause one on purpose, and a bug can cause one by accident. Defend with sender rate limits, bounded queues, filtering of noise and alerts on the pipeline itself.

Two ways it happens

Deliberate. RFC 5424 notes that attackers can flood collectors or fill disks, and suggests accepting messages only from known addresses and rate limiting. Triggering endless error paths in a public application does the same job from the outside. The goal may be denial of service, or cover: bury the one interesting event in a million boring ones.

Accidental. A retry loop that logs every attempt, a debug flag left on after a release, or a health check that logs each probe can produce more volume in an hour than a normal month.

The W3C Trace Context specification describes a related risk for tracing. A public service that blindly continues every incoming trace with the sampled flag set could face tracing overhead and higher bills, and it suggests rate limits and different handling for authenticated and unauthenticated requests.

What fails when the pipeline floods

• Disks fill, and processes crash or, worse, auditing stops.
• Collectors drop events, so the logs you needed are not there.
• Search slows down for everyone.
• The index budget runs out, and ingestion is throttled at the worst moment.
• Analysts drown in noise.

Controls, by layer

At the source

• Rate limit repeated identical messages and log a count instead.
• Cap the size of any single event and of user-controlled fields.
• Never log in a tight loop without backoff.

In transit

• Buffer on disk with a size limit and a clear overflow policy.
• Prefer dropping low value debug data over security events when the buffer is full. This needs priority classes.

At the collector

• Accept connections only from known senders and authenticate them. For OpenTelemetry's collector, the project's documentation advises being careful with buffer, queue, payload and worker settings, batching telemetry and setting memory limits to avoid out of memory errors. See the collector post.
• Use filtering to drop unneeded data before storage.

In storage

• Per source volume quotas, with an alert at a fraction of the quota.
• Retention tiers so old noise leaves expensive storage. See log tiers.

Protect security events from the noise

Separate streams for security events and diagnostic logs. If a flood hits diagnostics, the security stream keeps its own capacity and queue. This single decision does more than most tuning.

Alerts about the pipeline

1. Ingest rate several times its normal level for a source.
2. Disk usage on collectors above a threshold.
3. Drop or error counters rising.
4. Lag between event time and ingest time growing.

The last one links to detecting silent sources.

Rehearse it

In staging, flood a service with errors and watch what happens. Does the security stream survive? Does an alert fire? Fix the weakest link and repeat.

Quick check

Sources

• RFC 5424, The Syslog Protocol
• W3C Trace Context specification
• OpenTelemetry, Collector configuration best practices

Last reviewed: 19 October 2026.