Some models charge a higher per token rate once a prompt crosses a length threshold, and others charge one rate across the whole window. As of 19 October 2026, the thresholds differ by provider and model, so check the page for your model before you send very large prompts.
Three patterns on the public pages
Anthropic. The pricing page says Claude 4.6 and later models, except Claude Haiku 5.5, include the full 1M token context window at standard pricing, so a 900k token request is billed at the same per token rate as a 9k token request. Claude Haiku 5.5 is the exception: prompts over 100,000 tokens pay higher prices ($0.50 input and $2.50 output per million, against $0.10 and $0.50 up to 100,000). The page says prompt length counts all input tokens including cache reads and writes, and that each request is priced on its own.
Google. The Gemini pricing page lists two rates for some models. For Gemini 3.1 Pro Preview, input is $2.00 up to 200k tokens and $4.00 above, and output is $12.00 and $18.00.
OpenAI. The pricing page lists standard prices for short context, up to 272K input tokens, and notes that long context rates are higher for several models.
Why it matters
Large prompts are common in agent work. Each step of a coding agent can carry a long history, tool definitions and file contents. If your average request sits near a threshold, a small change in prompt size can change your rate for the whole request, not just the extra tokens.
How to find out which side you are on
- Count your real prompts with the provider's counter. See what is an AI token.
- Plot the distribution, not the average. A few huge requests can dominate the bill.
- Look at the share of requests above the threshold, and what share of tokens they carry.
Ways to stay under the line
- Retrieve instead of pasting. Send the relevant passages, not the whole file or archive.
- Summarize older conversation turns and keep recent turns verbatim.
- Split a large task into independent smaller calls.
- Drop tool definitions you do not use in that step.
- Cache the stable parts. Caching cuts price but, on Haiku 5.5, a cache read still counts toward prompt length.
Do not assume bigger is better
A model can accept a very long prompt and still answer better with a shorter, curated one. Test both on your own cases. A smaller input is cheaper on every provider, and it is faster.
Quick check
Does a 1M token window mean I should send 1M tokens?
No. It is a ceiling. Each token still costs money and adds latency, and relevance usually matters more than volume.
Do cached tokens count toward the threshold?
On Anthropic's Haiku 5.5, the page says yes, prompt length counts cache reads and writes. Check the rule for your model.
Sources
Last reviewed: 19 October 2026.
