An AI batch API accepts many requests in one submission, processes them asynchronously, and returns results later, usually within 24 hours. Both providers checked here document a 50 percent discount compared with synchronous calls. It suits work nobody is waiting on, such as evaluations, classification and bulk summaries.
How the flow works
You submit a file or list of requests. The service creates a batch, processes each request independently, and you poll for status or collect results when processing ends. The Anthropic documentation says most batches finish in under an hour, results are available when all requests complete or after 24 hours, whichever comes first, and batches expire if not done within 24 hours. OpenAI documents a 24 hour completion window as the only option currently, with batches often finishing sooner.
Requests that expire or are canceled before being sent are not billed, according to the Anthropic page.
What you trade for the discount
| You give up | You get |
|---|---|
| Immediate answers | A lower rate per token |
| Predictable per-request latency | A separate limit pool, per OpenAI's guide |
| Interactive use | Higher throughput for large jobs |
The Anthropic page also says processing may be slowed by current demand. So a batch is the wrong tool for anything interactive, and a poor fit for work with a hard deadline measured in minutes.
Limits to check
Documented limits differ by provider and change over time. At the time of review:
- Anthropic: a batch holds up to 100,000 requests or 256 MB, whichever is reached first, and results remain downloadable for 29 days.
- OpenAI: up to 50,000 requests per batch, input files up to 200 MB, and a cap on batch creations per hour.
Treat these as examples to verify, not constants.
Jobs that fit
- Re-running a fixed evaluation set after a prompt change.
- Classifying or tagging a backlog of documents or tickets.
- Generating summaries or embeddings for a content library overnight.
- Nightly analysis jobs where results are read in the morning.
Jobs that do not fit
- Anything a person is waiting for, including chat and code completion.
- Multi-step agent loops where each step depends on the last, since each round trip waits up to hours.
- Work that needs a guaranteed finish time.
Design tips
Give every request a custom ID so you can match each result to its input without relying on position. Handle partial failure, since individual requests can error while the batch succeeds. If your requests share a long prefix, the Anthropic page suggests the longer one hour cache duration because batches can take longer than the default five minute cache window to process. See how prompt caching works.
Discounts from different features can combine, so confirm in the pricing page how they stack before forecasting. For the idea of a slower, cheaper tier outside batch, see slower tiers that trade speed for cost.
Worked example: a nightly classification job
These figures are illustrative. A support team wants to tag 60,000 old tickets by topic. Nobody needs the tags until the next morning.
- Export the tickets and write one request per ticket, each with a custom ID equal to the ticket number.
- Split into batches that respect the provider's limits. For an example limit of 50,000 requests per batch, that means two batches.
- Add a shared instruction block at the top of every request so a cache can reuse it.
- Submit the batches in the evening and record the batch IDs.
- Poll for status every few minutes, or check in the morning.
- When the batches end, download results, match by custom ID, and write tags back to the tickets.
- Retry only the failed or expired requests in a new, smaller batch.
At the end, compare the job's metered cost against a synchronous run on a sample of 500 tickets. The documented discount is 50 percent on the providers checked, but your real saving also depends on caching and failures.
Batch versus synchronous
| Question | Batch | Synchronous |
|---|---|---|
| Someone waiting? | No | Yes |
| Typical finish | Often under an hour, up to 24 hours | Seconds |
| Rate limit pool | Separate | Your normal limits |
| Error handling | Per request, after the fact | Immediate |
| Best for | Backlogs, evaluations, bulk summaries | Chat, assistants, interactive tools |
Pitfalls
- Submitting a batch whose results you need for the next step of a pipeline that has a deadline.
- Forgetting that expired requests are not billed but also not done, so someone must notice.
- Mixing very different prompts in one batch, which makes failures harder to analyze.
- Not storing results before they age out. The Anthropic page says results stay downloadable for 29 days.
Monitoring a batch job
Treat a batch like any long-running job. Record the submit time, batch ID, request count and an expected finish window. Alert if a batch is still processing near the end of its window, since unfinished requests expire. After completion, compare the counts of succeeded, errored, canceled and expired requests with what you submitted, and fail the pipeline if they do not add up. Keep a small sample of results for human review each run, so a prompt that quietly degrades is caught before it is applied to tens of thousands of records.
Estimating whether batch pays off
A batch saves money only if the discount outweighs the cost of waiting and the added engineering. Estimate three numbers. First, the monthly spend on the work that could move to batch. Second, the discount on that spend, from the provider's current page. Third, the engineering time to add submission, polling and result matching, plus the operational cost of monitoring. For a large nightly backlog the saving is usually clear. For a handful of requests a day, the engineering may exceed the saving.
Also consider data handling. Check whether batch inputs and results are retained and for how long, using the provider's data retention pages, and make sure that fits your policy before you send sensitive material.
What this means for you
If you have work that nobody is waiting on, such as evaluations, backlogs or nightly summaries, a batch interface is worth a look. You give up speed and gain a lower rate and a separate limit pool. If your work is interactive, skip it.
Common mistakes
Typical errors are sending time-sensitive work to a batch, forgetting that unfinished requests expire, ignoring partial failures, mixing unrelated prompts in one submission so problems are hard to diagnose, and not saving results before they age out. Plan for per-request errors, match results by ID, alert on batches near expiry and retry only what failed.
Try it yourself
Click through it at your own pace, or open it full screen. It shows a backlog of findings being worked through in priority order. Notice how bulk work is organized by queue and status, which mirrors how you would track a large batch of requests.
Quick check
Is a batch result always ready in an hour?
No. Many finish quickly, but the documented guarantee is within 24 hours, after which unfinished work expires.
Can I use a batch for a chat feature?
No. Batches are asynchronous. Use them only when no one is waiting.
Does a batch use my normal rate limit?
OpenAI says batch has its own limit pool. Anthropic documents separate batch limits too. Check both pages for current numbers.
Sources
Last reviewed: 19 October 2026.
