Steady state is the measurable normal behaviour of your system, such as error rate, latency and the rate of blocked requests. A chaos experiment compares that baseline with what happens when you inject a fault. For security controls, pick metrics that show both user impact and whether the control still works.
What the principles say
The Principles of Chaos Engineering say to build a hypothesis around steady state, and to measure a system's observable output rather than its internals. Short-window metrics such as throughput, error rates and latency percentiles serve as proxies. The goal is to confirm the system works, by trying to disprove that steady state holds when something goes wrong.
That wording leaves a gap for security work. A fault in an authentication service might leave latency and error rates looking normal while the control quietly stops enforcing anything. So you need two kinds of metric.
Two metric families
Service health metrics. These tell you users are fine: request success rate, p95 and p99 latency, queue depth, saturation. They also act as your stop conditions.
Control effectiveness metrics. These tell you the control still does its job:
- Share of requests that were authenticated and authorised, compared with the share that bypassed a check.
- Denied requests per minute on a route that should deny a known bad input.
- Audit log events written per minute, against a floor.
- Alerts fired when a known test event is generated.
- Time from a revocation to the revoked credential being refused.
Without the second family, an experiment can pass while the system fails open. See fail open or fail closed.
Pick a baseline window
Capture the metric across normal variation first: weekday and weekend, peak and quiet. A baseline taken at 3am makes every 3pm result look like a failure. Write the range you consider normal, not a single number.
Avoid misleading proxies
- Averages hide tails. Use percentiles.
- A flat error rate can mean no traffic reached the control at all. Pair it with a volume metric.
- Metrics that come from the component under test go dark with it. Take at least one measurement from outside.
- Counting only successes misses a control that rejects everything.
Turn it into a hypothesis
State it in one sentence: when we make the identity provider unreachable for five minutes, login success stays above its baseline floor for cached sessions, and no request reaches the admin API without a valid token. The method is in how to write a chaos hypothesis for a security control.
Record the metrics before, during and after, plus the time to return to baseline.
Quick check
How many metrics should one experiment watch?
Few. Two or three health metrics for stop conditions and two or three control metrics for the hypothesis are enough.
What if there is no baseline yet?
Collect one for a couple of weeks before you inject anything. Running an experiment with no baseline proves nothing.
Sources
Last reviewed: 19 October 2026.
