Safeguard
Developer Tools•

When should an IoT fleet update halt itself? Success and error thresholds

Set two numbers per rollout group: a success level that unlocks the next group and an error level that stops the whole rollout. How to pick them.

Safeguard Security Team
3 min read

An IoT fleet update should halt itself when failures in any group pass a limit you set before the rollout starts. Pair that with a success threshold that unlocks the next group. Two numbers, written down in advance, remove the guesswork during an incident.

The two conditions

The Eclipse hawkBit documentation describes the cascade in these terms. The success condition is the percentage of successfully installed targets in the current group. When it is reached, a success action runs: start the next group, or pause. The error condition is an absolute number or percentage of failed installations, and when it triggers, the entire rollout shuts down as an emergency stop.

Those are the two controls worth copying, whatever tool you use.

Percentage or absolute count

StyleGood whenWeakness
PercentageGroups are largeA small group can trip on one device
Absolute countGroups are smallMeaningless as groups grow
BothYou want a floor and a ceilingMore settings to explain

For a first group of ten devices, one failure is ten percent. An absolute count of two is easier to reason about. For later groups, switch to a percentage.

What counts as failure

Be specific. A device that downloaded the image but has not yet rebooted is not a failure. These usually are:

  • The installer reported an error.
  • The device rolled back to the previous slot.
  • The device installed the update but failed its health check.
  • The device went silent past a stated time window.

The last one is the one teams forget. A bricked device reports nothing, so a silence timeout is the usual way it shows up as a failure.

Steps to choose values

  1. Start from what a bad outcome costs: truck roll, customer outage, safety impact.
  2. Set the error condition so the stop happens before that cost is large.
  3. Set the success condition high enough that a partial failure cannot slip through, and give devices a fixed time to report.
  4. Rehearse: run a rollout on test devices with a deliberately broken image and confirm the stop works.
  5. Write the thresholds into the release record.

After a halt

Stopping is the start of the response. Keep logs from the failing devices, pause new rollouts of any related release, and decide whether the fix is a new image or a rollback. Do not restart the same rollout with looser thresholds just to finish it.

If your pipeline scans the release SBOM before group 1, Safeguard can flag known vulnerable dependencies at that gate, so the halt rules deal with operational faults and not known issues. See gold.safeguard.sh/scan for the free scan.

Quick check

Should the error threshold be stricter than the success threshold?

They answer different questions. The error limit should be low enough to stop damage early. The success limit should be high enough to give you confidence in the next group.

How do I catch devices that fail without reporting?

Set a time window. A device that has not reported its new release and health by then counts as failed.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.