Stop bulk scraping by enforcing limits on the server: a maximum page size, caps on exports and batch operations, rate limits tuned to real use, minimal response fields and monitoring for enumeration. Authorization still comes first, because a limit only slows a caller who should not have had access.
What the risk looks like
OWASP API4:2023, Unrestricted Resource Consumption, lists records returned per page among the limits an API should set, and says to validate server side the parameters that control how many records are returned. If a client can send a page size of a million, one call can return a database. OWASP API3 adds that responses should return only what the endpoint needs.
Controls
| Control | What to set |
|---|---|
| Page size | A server side maximum, ignoring larger client values |
| Deep paging | A cap on offset or cursor depth where it fits |
| Fields returned | Only needed properties, no generic serialisation |
| Exports | Separate, authorised, rate limited, logged and sometimes asynchronous |
| Batch operations | A maximum number of operations per request |
| Rate limits | Per user and per key, stricter on sensitive endpoints |
| Search | Minimum query length, no wildcard only queries |
| Third party spend | Spending limits or billing alerts, as OWASP advises |
For GraphQL, OWASP's cheat sheet recommends combining depth and amount limits, pagination and query cost limits, and counting batched operations rather than only HTTP requests.
Enumeration signals
Scraping often shows up as patterns rather than single bad requests:
- one client walking IDs in order
- many distinct object IDs requested by one identity in a short time
- repeated list calls with maximum page size
- requests spread across many addresses with the same token
- access outside normal hours or at machine speed
OWASP's API6 text names non human patterns as a key signal. Log requester, object type and count, then alert on thresholds. See what an API should log.
Steps
- Set a default and a maximum page size in one shared function.
- Return a clear 400 when a request exceeds the maximum, or clamp it, and document which.
- Review each list endpoint for fields that are not needed.
- Move large exports to a dedicated, audited flow.
- Add per identity quotas, not just per address.
- Alert on enumeration patterns.
- Test with a script that requests huge pages and walks IDs.
Do not forget the owner check
A cap makes scraping slower and noisier. It does not prevent a user from reading records they should not see. Pair limits with the checks in do random UUIDs stop IDOR.
Quick check
Should I clamp the page size or return an error?
Either is acceptable. An error is clearer for client developers. Clamping is friendlier to old clients. Document the choice.
Will rate limiting alone stop scraping?
No. Patient scrapers stay under limits. Combine limits with minimal responses, authorization and monitoring.
Safeguard policies and gates can enforce checks on changes to such endpoints, and the free scan at gold.safeguard.sh/scan is a quick start.
Sources
- OWASP API4:2023 Unrestricted Resource Consumption
- OWASP API3:2023 Broken Object Property Level Authorization
- OWASP API6:2023 Unrestricted Access to Sensitive Business Flows
- OWASP GraphQL Cheat Sheet
Last reviewed: 19 October 2026.
