When the datastore behind a rate limiter is unreachable, choose per route. Fail closed or use a strict local limit on login, password reset and costly endpoints. Fail open on low-risk reads, with an alert. A per instance fallback limit is usually better than either extreme.
Why the question comes up
Many rate limiters keep counters in a shared store so every instance sees the same totals. When that store is slow or down, each request hits a decision: allow it without a count, or reject it because you cannot count.
Neither is free. Failing open means the limit vanishes at the moment you are weakest. Failing closed means a counter outage becomes a full outage of every limited route.
Route by route
| Route type | Suggested behaviour | Why |
|---|---|---|
| Login, password reset, MFA verify | Local strict limit, else closed | Credential stuffing is the main threat |
| Payment or money movement | Closed or local strict limit | Abuse costs real money |
| Expensive queries, exports, LLM calls | Local limit | Cost and capacity risk |
| Public read endpoints | Open, with alert | Availability matters more |
| Health checks and static assets | Bypass the limiter entirely | They should not depend on it |
For LLM calls, abuse is also a billing issue, see denial of wallet.
The local fallback
Keep a small in-process counter per instance. When the shared store fails, switch to it with a limit set to your normal global limit divided by the expected instance count, or lower. It is imperfect, since an attacker can spread requests across instances, but it keeps a ceiling in place and needs no network.
Switch back when the store recovers, and avoid a flood of resets. A limiter that wipes its counters at recovery hands every client a fresh allowance.
Tight timeouts
The store call must have a short timeout, a few milliseconds to tens of milliseconds depending on your latency budget. A slow limiter that adds a second to each request is worse than a missing one. Do not retry at every layer, see retry storms, and consider a circuit breaker around the store so you switch to the fallback quickly.
Alert and record
A limiter running in fallback mode is a degraded security control. Alert when it happens, log the duration, and review afterwards. If you run this experiment on purpose, record how long the switch took and what traffic got through. This fits the method in how to write a chaos hypothesis for a security control.
Quick check
Is failing open on a rate limiter always wrong?
No. For low-risk routes, keeping the service available can be the better trade, as long as you alert and the decision was deliberate.
Do I need a limiter in front of health checks?
No. Exempt them so orchestrators do not restart healthy instances because the limiter is down.
Last reviewed: 19 October 2026.
