A heartbeat alert fires when something you expect to happen regularly stops happening. It catches the failures that normal alerts cannot, because a dead job, a silent log source or a broken alert pipeline produces no data that could cross a threshold. Absence itself is the signal.
The blind spot
Most alerts say "if this value is bad, tell me". If the system emitting the value dies, nothing is bad. The graph is empty and the alert stays green. Many teams learn this only when a backup has not run for weeks.
Three places to use heartbeats
- Scheduled jobs. Each run reports success to a checker. If no success arrives within the expected period plus a grace window, alert. This beats alerting on job errors, because a job that never started raises no error.
- Log and metric sources. Track events received per source. A drop to zero or a sharp fall means an agent stopped, a shipper broke or a service is down. OWASP's guidance on protecting logs includes availability attacks, such as blocking writes, which this also helps detect.
- The alerting pipeline. Send a constant test alert, often called a dead man's switch, to an external receiver. If the receiver stops hearing it, the pipeline is broken, and the receiver pages you through a separate path.
Design choices
- Pick the period from the real schedule. A nightly job gets a window of a day plus slack, not five minutes.
- Add a grace period for slow runs and clock differences.
- Report success, not just start. A job that starts and crashes should not count.
- Use a separate system for the external receiver, so one outage does not silence both.
What to do when it fires
Link a runbook that says what to check first: is the scheduler running, are credentials valid, is the network path open, is the disk full. Include how to run the job by hand and how to confirm that it worked.
Do not overdo it
Heartbeats on trivial or experimental jobs create the same fatigue as any noisy alert. Reserve them for jobs and sources whose silence would hurt: backups, certificate renewals, security log feeds, billing runs and data exports.
Security angle
Attackers who gain access sometimes stop or tamper with logging to hide activity. A heartbeat per host that sends logs gives you a chance to notice. Treat a silent security log source as an event to investigate, not only an ops problem.
Quick check
Can I build this myself?
Yes. A scheduled job that posts to a checker URL on success, and a checker that alerts if the last post is too old, is a few dozen lines.
How is this different from an uptime check?
An uptime check probes a service from outside. A heartbeat waits for a service or job to report in on its own.
Sources
Last reviewed: 19 October 2026.
