Page people on symptoms that users feel, such as failed requests or slow responses, and keep causes such as high CPU or a full queue on dashboards for debugging. Cause alerts fire when nothing is wrong, and they miss failures you did not predict.
Symptom versus cause
The Google SRE book frames monitoring as two questions: what is broken, and why. The "what" is the symptom. The "why" is a possibly intermediate cause. It says symptoms are better for paging and that causes mainly help debugging.
Some pairs make it concrete.
- Symptom: checkout returns errors for 4 percent of requests. Cause: the payments database connection pool is exhausted.
- Symptom: search takes eight seconds. Cause: a cache node restarted and the cache is cold.
- Symptom: password reset emails are not arriving. Cause: the mail provider is throttling.
Why cause alerts fail
A CPU alert at 90 percent might be a healthy batch job. A disk alert might fire a week before it matters. Worse, the list of possible causes never ends. You cannot write an alert for every way a system breaks, but you can detect the one outcome that matters: users are not getting what they need.
Cause alerts also train people to ignore pages. When most pages need no action, the real one is treated like the rest.
What a good page looks like
The SRE book gives the test. A page should be urgent, actionable and need human judgment. Ask three questions:
- does this reflect real user impact?
- can the person on call do something about it?
- could it wait until morning?
If it could wait, send it to a ticket queue or a chat channel, not a phone.
What to keep as cause signals
Do not delete cause metrics. They are the first thing you open after a page. Put them on a dashboard next to the symptom graph so the on call person can see the likely reason within a minute. A few can stay as non-paging warnings, such as a disk that will fill within days.
Exceptions worth keeping
Some causes are worth paging when the symptom would arrive too late to act on. Expiring certificates, a backup that has not completed, and a near-full disk on a database with no headroom are examples. Make these specific and rare, and attach a runbook.
A short exercise
List every alert that paged in the last month. Mark each as symptom, cause or noise. Convert the repeat offenders in the cause and noise groups into dashboard panels, and replace them with one or two symptom alerts on error rate and latency.
Quick check
Is a high error rate always a symptom alert?
Yes when it measures what users get back, such as failed requests. An error count inside a background worker that no user waits on may not need a page.
Should security events page people?
Some should, such as signs of an account takeover in progress. Apply the same test: urgent, actionable and needing judgment.
Sources
Related: tune or suppress a noisy alert.
Last reviewed: 19 October 2026.
