AI services bill for tokens processed, cap use with rate limits and spend limits, and offer discounts for caching and batch. Subscriptions replace token meters with usage windows. This page gives short answers to ten common questions, each tied to provider documentation and linked to a longer guide.
What is inference?
Inference is running a trained model to produce an answer, as opposed to training it. Each request you send is an inference call, and hosted providers meter it. Self-hosting means you run the hardware that performs it.
What is a token?
A token is a chunk of text, often a word piece, that the model reads or writes. Bills and limits count tokens. Different models can split the same text differently, so counts vary. See what is a token.
Why do bills have separate input and output meters?
Reading a prompt and generating a reply are different work, and providers price them separately. OpenAI's latency guide says generating tokens is almost always the highest latency step. See input vs output tokens.
What counts as input?
Everything in the request: system prompt, history, tool results, tool definitions, images and documents. The Anthropic context documentation lists these. See what counts toward a context window.
Why does a long chat get pricier?
History is resent every turn, so each turn carries all earlier ones. Caching can soften the repeated part. See how prompt caching works.
Are subscription plans unlimited?
Not the one documented here: it has a five-hour session limit and a weekly limit shared across apps and the coding tool. See why subscriptions have usage limits.
What is a rate limit?
A cap on requests and tokens per period, enforced per organization, with continuous refill under a token bucket on at least one provider. See what is an AI API rate limit.
What is a spend cap?
A maximum cost per period. Reaching it can pause usage, and retries will not help until it resets or is raised. See 429 rate limit or spend cap.
When is batch worth it?
When nobody is waiting and results can arrive within a day. Both providers checked document a 50 percent discount versus synchronous calls. See how batch APIs work.
Is self-hosting cheaper?
It depends on utilization, staffing and security work. Build a worksheet with your own numbers. See self-hosted or hosted cost comparison.
A small glossary
| Term | Meaning |
|---|---|
| RPM | Requests per minute |
| ITPM and OTPM | Input and output tokens per minute |
| Cache read | Input served from a reused prefix |
| Context window | Everything the model can reference in one request |
| Tier | A bundle of limits tied to account history or spend |
How to turn these answers into a one page brief
Leaders rarely want the whole guide. Build a one page brief with four blocks.
- Meters: which of input, output, cache, batch, region and tools apply to us.
- Controls: spend limit, workspace limits, key ownership and alerts, with owners.
- Limits: our tier, per model limits and the date checked.
- Tradeoffs we chose: where we accept slower or cheaper tiers, and where we pay for speed.
Worked example: filling the brief
Suppose a team uses one chatbot, one nightly job and a small experiments area. The meters block lists input, output, and cache reads on the chatbot, plus a batch discount on the nightly job. Controls list a spend limit set below the tier cap, a workspace per service, and a daily spend alert. Limits list the tier and one line per model. Tradeoffs record that the nightly job runs on batch because nobody waits for it, and that the chatbot stays on standard because delay is the product.
That brief takes an afternoon to write and answers most questions finance and security will ask.
More questions people ask
| Question | Short answer | Read next |
|---|---|---|
| Why was my bill higher than last month with the same users? | Longer prompts, longer outputs, a model change or a cache miss | bill spike runbook |
| How do I forecast spend? | Tokens per task times tasks, with low, expected and high cases | pricing page dimensions |
| Do I need a separate key per service? | Yes, for attribution and limits | workspace limits |
| Can limits protect me from a leaked key? | They bound the damage, they do not prevent misuse | spend cap as a control |
| What if I hit a limit mid launch? | Queue, cache, shrink, or request an increase early | usage tiers |
Keep it current
Pricing pages, tiers and limits change. Add a recurring calendar item each quarter to reread the pages you cited and update the brief.
Who should own the brief
Pick one person, often an engineering manager or a platform lead, to own the one page brief, and name a finance contact and a security contact as reviewers. Ownership matters because limits, prices and models all change. Without a name on it, the brief goes stale within a quarter and the next surprise arrives anyway.
Where this page stops
This page gives short answers and points to longer guides. It does not give prices, and it does not tell you which provider or plan to choose. Those decisions depend on your workload, your data rules, your team and your budget, and they need your own measurements. Use the pages cited in each linked guide as the starting point, and put your own numbers in a dated worksheet.
What this means for you
If you only remember a few things from this page, make them these. Bills follow meters, and there are several meters. Limits come in two kinds, one about speed and one about money, and they fail in different ways. Caching and batch can lower cost, but only when your workload fits them. And a subscription is a different kind of product from a metered API, so do not compare them by price alone.
Beyond the facts, the habit that helps most is measuring. A single week of real usage data answers questions that no amount of reading can. It tells you how long your prompts are, how long your answers are and how often you hit limits.
Common mistakes
People often compare a headline rate per token across providers without checking how many tokens the same task needs on each. They plan on averages and get surprised at peaks. They set no spend limit because the first month was cheap. They retry every error, including the ones that cannot succeed. They put every service on one key and then cannot tell which one caused a spike. They treat an old blog post as current documentation. Each of these is cheap to fix once you notice it.
A step by step first week
- Day one: list every place your organization calls an AI service, and who owns each.
- Day two: create separate keys and workspaces, and set a spend limit below the tier cap.
- Day three: turn on usage reporting and a daily spend alert.
- Day four: measure tokens per task for your two most important workloads.
- Day five: write the one page brief described above and share it with finance and security.
By Friday you will know more about your AI costs than most teams do after a quarter of drifting, and you will have the controls in place before a surprise arrives.
Try it yourself
Click through it at your own pace, or open it full screen. It shows a prioritized list that a team works through with owners and status. Use the same shape for your AI cost brief: each item has an owner, a status and a next review date.
Quick check
Where should I confirm any number in this page?
On the provider's current documentation and pricing pages. Terms change, and this page reflects its review date only.
What is the first thing to measure?
Tokens per completed task for one real workload, split into input, output and any reasoning.
What is the first control to set?
A spend limit below your tier cap, plus separate keys per service.
Sources
- Rate limits, Claude API documentation
- Rate limits, OpenAI API documentation
- Batch API guide, OpenAI
- Batch processing, Claude API documentation
- Latency optimization, OpenAI API documentation
- Using the coding tool with your Pro or Max plan, Claude Help Center
- Context windows, Claude API documentation
Last reviewed: 19 October 2026.
