Safeguard
Developer Tools•

Head or tail sampling: which keeps the traces security needs?

Head sampling decides before a trace is complete and can drop errors. Tail sampling decides after. How each affects security evidence, with a table.

Safeguard Security Team
4 min read

Tail sampling keeps the traces security cares about more reliably, because it decides after seeing the whole trace. Head sampling decides at the start, so it can drop a trace that later turns out to contain an error or an attack. Many teams combine both.

Definitions

The OpenTelemetry sampling documentation separates the two approaches.

Head sampling makes its decision as early as possible. It does not inspect the trace as a whole. The most common form uses the trace ID and a target percentage, so a fixed share of whole traces is kept. It is easy to configure and cheap to run. Its limit, in the documentation's words, is that it cannot make a decision based on data in the entire trace. For example, it cannot guarantee that every trace containing an error is kept.

Tail sampling decides after considering all or most of the spans in a trace. You can keep traces based on errors, total latency, specific attributes, or the volume of the originating service. The cost is complexity: it is harder to implement, needs stateful components that handle large data volumes, and its rules must change as your systems change.

Why this matters for security

Security evidence is rare by nature. If 1 percent of traces are kept at random, then a single failed login burst or one suspicious admin call has a 99 percent chance of vanishing. When you later investigate, the trace you need was never stored.

Sampling is not only a cost question. It decides what you can prove afterwards.

NeedHead samplingTail sampling
Keep a steady, representative sliceGoodGood
Keep every trace with an errorCannot guaranteeCan
Keep traces for admin or privileged routesOnly if the rule uses early attributesYes
Low operating effortYesNo
Reconstruct an incident timelineWeakBetter

Steps to decide

  1. List the five questions you would ask during an incident, such as which requests touched a given account, or which calls failed authorization.
  2. For each, ask whether a random sample would still answer it. If not, that traffic must be kept in full.
  3. Use a separate path for security events. Audit events and authorization failures belong in logs that are never sampled. See what to log in a web app and what never to log.
  4. If you use head sampling for volume, apply it only to healthy, high-volume routes.
  5. If you need error and latency based retention, plan for tail sampling and budget for the stateful collectors it requires.
  6. Document what is sampled and at what rate. An investigator, or an auditor, should not have to guess.

The combined approach

The documentation notes that in very high-volume pipelines you can apply head sampling first to cut volume, then tail sampling later for refined decisions before export. The caution: the head stage still drops traces blindly, so only use it where losing a random trace is acceptable.

A related trap

Sampled traces still contain whatever attributes you attached. Fewer traces does not mean less sensitive data per trace. Redact first. See redact at the source or in the pipeline.

Quick check

Can head sampling guarantee I keep every error trace?

No. It decides without seeing the whole trace, so an error that occurs later in the request cannot influence the decision.

Should audit events be sampled?

No. Record security and audit events outside the trace sampling path so every one is kept.

Safeguard's dependency scanning and SBOMs show which telemetry libraries you ship, which helps when a collector component needs an update. A free scan is at gold.safeguard.sh/scan.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.