A rate limiter that counts the wrong thing does not fail loudly. It runs, it logs, it appears in your architecture diagram, and it permits everything.
There are four ways to get this wrong and they are all silent. This post is the four, and how to test for each in a way that actually distinguishes a working limiter from a decorative one. For whoever owns a public endpoint that costs something to serve.
One: the key is attacker-controlled
Covered at length elsewhere, and it is the most common failure so it belongs here too. If your bucket key comes from a request header the caller can set, the caller gets a fresh bucket whenever they want one.
The usual culprit is the first entry of X-Forwarded-For, because proxies append to that header rather than replacing it, so the leftmost value is whatever the client typed. Parse from the right, using a hardcoded count of trusted proxies.
The same applies to any other identity you key on. A user id from a JWT is fine, because it is signed. A user id from a header is not. An API key is fine. A User-Agent is not.
Two: the limit is per instance
You run six application instances behind a load balancer. Each holds its own in-memory counter. Your advertised limit of 100 requests a minute is actually 600, and it drifts as you scale.
Worse, it is not even a consistent 600: a caller whose requests happen to land on one instance gets limited at 100 while another caller spread evenly gets 600. Behaviour depends on load balancer hashing, which is not something you can explain to a customer.
The fix is shared state, usually Redis, with an atomic increment:
INCR ratelimit:{key}:{window}
EXPIRE ratelimit:{key}:{window} 60 NX
Two things to get right. The increment and the expiry must not race, so set the expiry only when the key is new (NX) or use a Lua script to make it one operation. And decide what happens when Redis is unavailable, which is the next failure.
Three: it fails open, silently
Redis goes down. The limiter cannot read a counter. What does your code do?
The common implementation catches the exception, logs a warning, and allows the request, because nobody wants a cache outage to become an availability outage. That is a defensible decision and it is frequently made implicitly, by a try/catch written without anyone deciding anything.
The consequence: an attacker who can degrade your Redis has disabled your rate limiting, and the only signal is a warning in a log nobody reads.
Decide deliberately, and make the decision visible:
- Fail open for limits protecting cost or fairness. Add a metric, alert on it, and treat sustained fail-open as an incident rather than a warning.
- Fail closed for limits protecting security: login attempts, password reset, token issuance, anything enumerable. A brief outage of a login endpoint is better than an unlimited credential stuffing window.
The important part is that the two categories get different code paths. A single shared limiter with one failure mode will be wrong for one of them.
Four: the window is wrong for the attack
A fixed window of 100 per minute permits 200 requests in two seconds: 100 at 11:59:59 and 100 at 12:00:00. For a limiter protecting an expensive endpoint, that burst is the entire problem.
Sliding window or token bucket fixes it. Token bucket also expresses the thing you usually want, which is a sustained rate with a tolerated burst:
capacity 100, refill 100 per minute
→ a burst of 100 is fine, sustained 200/min is not
Separately, match the window to what you are protecting. Login attempts need a long window, because a patient attacker at five attempts an hour defeats a per-minute limit entirely. Expensive queries need a short one. One global limit cannot serve both, and most systems that have one global limit have chosen the wrong window for at least one endpoint.
Testing it properly
The standard test is to hammer the endpoint from one machine and watch for 429s. That test passes on a limiter broken in three of the four ways above.
Test each failure specifically:
# 1. Key forgeable? Rotate the header. A correct limiter still blocks.
for i in $(seq 1 200); do
curl -s -o /dev/null -w "%{http_code} " \
-H "X-Forwarded-For: 198.51.100.$((i % 254 + 1))" https://api.example.com/endpoint
done
# 2. Per instance? Send the limit exactly, repeatedly, and count total accepted.
# More than the limit means the counter is not shared.
# 3. Fails open? Stop Redis in staging and repeat test 1.
# Everything 200 means you fail open. Was that the decision?
# 4. Window boundary? Send the full limit just before a window boundary
# and the full limit just after.
Test four is the one nobody runs and it takes thirty seconds with a sleep.
What to emit
A limiter you cannot observe is one you cannot trust:
- A counter of limited requests, by endpoint and by reason.
- A counter of fail-open events, alerted, as described above.
- The remaining allowance in response headers (
RateLimit-Remaining), because it turns a customer support conversation into a self-service answer. - The bucket key, hashed, in the log line. During an investigation you need to know what the limiter thought the caller was, and that is exactly the field that reveals a forgeable key.
The concession
Rate limiting is not a security control in the way access control is. A determined attacker with a botnet has many addresses, and per-address limiting is close to useless against them. Anything genuinely valuable needs authentication, proof of work, or anomaly detection rather than a counter.
What a limiter buys is protection against the cheap attacks and against your own users, which is most of the volume most of the time. Judge it on that, and do not let its presence stand in for a control against a serious adversary.
The implication
Every failure here produces a limiter that runs and permits everything, and none of them shows up in a normal test. That combination is why these survive for years in production systems.
So test the failures rather than the happy path. Rotate the header, stop the cache, cross the window boundary. Each takes a minute, and each distinguishes a control from a decoration.
