Rate limiting caps how much a single client or key may send, regardless of how busy the service is. Load shedding rejects some work when the service itself is near its limit, regardless of who sent it. Rate limits enforce fairness and abuse control. Load shedding protects the service from collapse.
Two different questions
Rate limiting asks: is this caller sending more than their share? Load shedding asks: can this service take more work right now?
A service can fail either way. One abusive client can burn through capacity while everyone else stays polite, and a rate limit handles that. Or a hundred polite clients can arrive together after a deploy, each within their limit, and the sum is too much. Only load shedding handles that.
What load shedding looks like
The Google SRE book describes it as rejecting excess work early and cheaply, for example returning HTTP 503 once the number of in-flight requests passes a threshold. It also suggests capping queue lengths and considering LIFO or CoDel queuing so stale requests are dropped rather than served after the caller has given up.
The point is that rejecting a request costs almost nothing, while accepting it and failing slowly costs a thread, memory and a held connection. A service that sheds load keeps serving the requests it accepts. One that does not tends to slow down for everyone and then crash.
Which to use when
| Situation | Control |
|---|---|
| One key sends far too much | Rate limit |
| Credential stuffing against a login route | Rate limit, per account and per source |
| Sudden spike from many normal clients | Load shedding |
| Dependency slows and requests pile up | Load shedding, plus timeouts |
| Costly endpoints (search, export) | Both, with a tighter per-client limit |
Decide who gets shed
If you must drop work, decide in advance what survives. Health checks, authentication and payment completion usually rank above reports and background sync. Tag requests with a priority so the shedder has something to go on. A shedder that drops at random can reject the one request you needed.
Security notes
Shedding helps with denial of service, but it must not open a hole. Never shed by skipping authorisation checks to save time. Make sure error responses do not leak internal state, and make sure clients get a clear retry signal, ideally with a Retry-After header, so they do not hammer you. For how clients should react, see handling 429 and Retry-After in API clients.
Test it. The SRE book recommends finding each component's breaking point under load and confirming it sheds load rather than crashing, including sudden and gradual increases and recovery back to normal load.
Quick check
Is a 429 the same as a 503?
Usually not. A 429 tells one client it exceeded its own limit. A 503 says the service cannot take the work right now, whoever sent it.
Can I rely on a CDN or gateway alone?
It helps at the edge, but it cannot see your internal saturation. Add in-service limits on in-flight requests too.
Sources
Last reviewed: 19 October 2026.
