Often, yes, and you may not know until a bad day. If your runbooks, chat, remote access or login rely on the same network, DNS or identity system that failed, recovery stalls. Map what each recovery step needs, and keep a path that does not share the failure.
What a published report shows
Meta's engineering team published an account of the October 4, 2021 outage. It says a command issued during routine maintenance, meant to assess backbone capacity, unintentionally took down all connections in the backbone network. An audit tool meant to stop such commands had a bug.
The report then describes a second effect. The company's DNS servers withdraw their network route advertisements if they cannot reach the data centers, as a sign of an unhealthy connection. With the backbone gone, they did so, and the DNS servers became unreachable even though they were still running.
The part that matters for this post is the recovery. The report says engineers faced two obstacles. It was not possible to access the data centers through normal means because their networks were down, and the total loss of DNS broke many of the internal tools normally used to investigate and resolve outages. Primary and out-of-band network access was down too, so engineers were sent onsite, which took time because the facilities are built with strong physical and system security.
The author also notes that security hardening slowed recovery, and that the tradeoff was considered worthwhile.
The pattern
A recovery path that depends on the failing system is a circular dependency. It is easy to create by accident, because tools are built and tested while everything works.
| Recovery need | Common hidden dependency |
|---|---|
| Reach the runbook | Wiki hosted on the same cloud account or network |
| Talk to the team | Chat that logs in through the same identity provider |
| Log in to fix things | Single sign-on that needs the system that is down |
| Reach servers | Remote access that resolves names through your own DNS |
| Deploy a fix | Pipeline that pulls from your own registry and signs with a key stored in the same vault |
| Know what is happening | Monitoring hosted on the infrastructure it watches |
An afternoon exercise
- Pick one scenario, such as loss of your primary network, DNS or identity provider.
- List the first ten steps your team would take to recover. Include finding the runbook and reaching each other.
- Beside each step, write what it needs in order to work.
- Mark any step whose need is affected by the scenario you picked.
- For each mark, add an alternative: an offline copy of the runbook, a phone tree, a local account with a stored credential, a direct IP address list, a secondary status page host.
- Test one alternative this week. Confirm that the offline copy opens without your corporate login.
Keep break-glass credentials protected, logged and tested, as in our post on break-glass credentials.
What not to conclude
Do not respond by weakening access controls. The report is clear that the hardening was a deliberate choice. Aim to remove circularity and rehearse the physical or manual path, not to remove protections.
A worked example: a mapping table for a network and DNS failure
This example is hypothetical. A team picks the scenario of losing its primary network and internal DNS. It writes the first ten recovery steps and what each needs.
| Step | Needs | Affected by scenario? | Alternative |
|---|---|---|---|
| Find the runbook | Wiki on the corporate network | Yes | Printed copy, encrypted offline file |
| Alert the team | Chat via company sign in | Possibly | Phone tree, separate messaging provider |
| Join a call | Video service via company sign in | Possibly | Conference line |
| Log in to a router | VPN, internal DNS | Yes | Out-of-band console, local account |
| Reach a server by name | Internal DNS | Yes | Hosts file with key addresses |
| Read monitoring | Hosted on the same network | Yes | External synthetic checks |
| Check the status page | Hosted by a third party | No | None needed |
| Deploy a fix | Pipeline needing internal registry | Yes | Offline build bundle |
| Notify customers | Email via company domain | Possibly | Third party status page |
| Verify recovery | Monitoring and test accounts | Yes | External checks and a test login |
Seven of ten steps are affected. Most of them have a simple alternative, and the work to prepare them is measured in days, not months.
Step by step: closing the gaps
- Print the table and assign each alternative to an owner with a date.
- Start with the runbook and the contact list, since everything else depends on finding and reaching people.
- Build the offline copies and test that they open on a laptop with no company network access.
- Add the out-of-band path for network devices, such as a console server on a separate connection, and record who may use it and how it is logged.
- Keep a short list of critical addresses that do not depend on DNS, stored with the offline runbook.
- Add external monitoring that does not run inside the network you monitor.
- Rehearse with a tabletop, then with a real disconnected drill in a safe window.
Hardening and recovery speed
Meta's account says that hardening against unauthorized access slowed engineers during recovery, and that the author considered the tradeoff worthwhile. That is a helpful framing. Strong access controls are not the enemy of recovery. They make it important to plan the recovery path, including who can enter secure facilities or use break-glass accounts, how long it takes, and how the access is audited.
Do not remove controls to speed recovery. Instead measure how long your emergency access takes, and shorten it by preparation: pre-approved people, tested procedures, and credentials that are stored safely and checked on a schedule.
Related dependencies people forget
- Time. If clocks drift, tokens and certificates fail.
- Licenses and activation checks that need an outside service.
- Password managers that sit behind the same sign in as everything else.
- The ticketing system where the incident record lives.
- Contracts and contact details for vendors you may need to call.
For each, ask whether you can reach it when your normal path is gone.
Try it yourself
Open the zero day watch demo and look at how alerts are raised and where they go. Ask the same question of your own alerts: if your usual channel is down, how would someone still learn that something has happened?
Click through it at your own pace, or open it full screen.
Quick check
How do I find circular dependencies?
Walk through a recovery step by step and ask what each step needs. Then ask whether the scenario you are planning for could remove that need.
Where should an offline runbook copy live?
Somewhere that opens without your usual login or network, such as printed copies or encrypted storage that several people can reach. Review it on a schedule.
How often should this exercise be repeated?
Repeat it after major infrastructure changes and at least once a year. New tools and vendors tend to add new dependencies without anyone noticing.
Sources
Last reviewed: 19 October 2026.
