Why AI inference demand keeps growing: reasoning, agents, usage

Inference demand grows because three things multiply: how many people use AI, how often they use it, and how many tokens each task consumes. Reasoning models and agents raise the third factor sharply. Cheaper tokens lower the cost of each request, but they also make new uses worth building, so total demand keeps rising.

Key numbers at a glance

• Over 3.2 quadrillion tokens a month: processed across Google's surfaces, up about 7 times from roughly 480 trillion a year earlier (Google, 19 May 2026; company statement).
• About 19 billion tokens per minute: through Google's model APIs, against more than 16 billion in late April and 10 billion the quarter before (Google, 19 May 2026; Alphabet Q1 call, 29 April 2026).
• 900 million weekly users: of ChatGPT, up from 800 million in October 2025 (OpenAI figures reported by TechCrunch, 27 February 2026).
• 25 trillion tokens a week: on the OpenRouter model marketplace, five times the 5 trillion of six months earlier (OpenRouter press release, 26 May 2026).
• About 1,500 to over 6,000: average prompt tokens per request on OpenRouter; completions rose from about 150 to 400 tokens (a16z and OpenRouter study, December 2025).
• Over 50 percent: reasoning models' share of OpenRouter tokens by late 2025, from a negligible slice early that year (a16z and OpenRouter study, December 2025).
• About 4x and 15x: tokens used by agents and by multi agent systems, compared with chat interactions, in Anthropic's research feature (Anthropic engineering post, 13 June 2025).
• 80 percent: share of performance variance on the BrowseComp benchmark explained by token usage alone in the same Anthropic work (13 June 2025).
• 4x: year on year growth in Microsoft Foundry customers running at a 1 trillion token annualized rate (Microsoft earnings call, 29 July 2026).
• About 13x per year: fall in the cost of reaching a fixed benchmark score since 2023 (Epoch AI, 22 September 2026).

Demand is a multiplication

Total inference demand is roughly the number of requests multiplied by the tokens per request. A token is a small chunk of text, often part of a word. Requests depend on how many people have access, how many features call a model, and how often each feature fires. Tokens per request depend on prompt length, carried history, hidden thinking and the length of the answer.

That framing shows where growth comes from even when one factor stalls. If user numbers flatten, demand still climbs if each task uses more tokens. If efficiency cuts tokens per task, demand still climbs if requests multiply. Most of the drivers below act on one factor, and several act on both.

The main public evidence for the total comes from companies' own usage disclosures. Google said at its I/O conference on 19 May 2026 that it processes over 3.2 quadrillion tokens a month across its surfaces, up about 7 times in a year and up from 9.7 trillion a month two years earlier. Alphabet said on its 29 April 2026 earnings call that its models handled more than 16 billion tokens per minute through direct customer API use, up from 10 billion the previous quarter; three weeks later Google put the figure at roughly 19 billion. These are self reported, unaudited usage numbers, and companies define tokens processed in their own way. No audited worldwide total exists.

Driver one: more people and more places

The simplest driver is reach. OpenAI said in February 2026 that ChatGPT had 900 million weekly active users and 50 million paying subscribers, as reported by TechCrunch on 27 February. Google said in May that its Gemini app passed 900 million monthly active users and that AI Overviews in Search had over 2.5 billion monthly users.

Reach is not only chat. A 2021 paper by Carole-Jean Wu and co-authors at Meta reported trillions of inferences per day across the company's data centers, more than doubling over three years, mostly from ranking and recommendation models working behind every feed view. Any product that scores, ranks, summarizes or filters content on every page view adds requests without a visible chat window.

Driver two: thinking before answering

The second driver is reasoning models, which spend tokens working through a problem before they reply.

Research shows why labs went this way. A paper by Charlie Snell and co-authors (August 2024) found that adapting the amount of answer time compute to each prompt improved efficiency by more than 4 times over a best of N baseline. In a comparison matched on total operations, a smaller model given extra compute at answer time could outperform a model 14 times larger on problems where the smaller model already had a non trivial success rate.

If better answers come from thinking longer, better answers mean more tokens. The a16z and OpenRouter study, which covered more than 100 trillion tokens over 13 months to November 2025, found reasoning models went from a negligible slice of traffic in early 2025 to more than half of all tokens. Stanford's 2025 AI Index gave a cost example from the first wave: o1 was nearly six times more expensive and 30 times slower than GPT-4o.

Provider documentation shows how this reaches the bill. Anthropic's documentation says thinking tokens are billed as output tokens, that the billed output count does not match the visible text, and that max_tokens is a hard cap on thinking and answer combined. It says that at high effort and above the model may think extensively. Google's Gemini pricing page lists output prices as including thinking tokens. See metering thinking tokens.

Driver three: agents that loop

A chat answer is one request. An agent is many. An agent takes a goal, picks a step, calls a tool, reads the result and decides the next step, repeating until done. Each pass is a model call, and each call carries instructions, tool definitions and the history so far.

The clearest published measurement comes from Anthropic's engineering team, writing on 13 June 2025 about its multi agent research feature. It reported that agents typically use about 4 times more tokens than chat interactions, and multi agent systems about 15 times more. It also found that token usage alone explained 80 percent of the variance in performance on the BrowseComp browsing benchmark, and that token usage, tool calls and model choice together explained 95 percent. In other words, spending more tokens was the main way the system got better answers.

Anthropic's documentation describes two more effects. Thinking can interleave with tool calls, so the model reasons after each tool result. And in a tool use loop each request has its own max_tokens, so that cap does not bound the spend of the whole turn. Its documentation also says newer models keep earlier turns' thinking blocks in context and bill them as input tokens.

Enterprise adoption shows the same direction. Microsoft said on 29 July 2026 that Agent 365, two months after launch, had nearly 40 million agents registered across tens of thousands of companies. That is a count of registered agents, not active usage, but every active agent is a source of repeated model calls. Prompt caching exists to soften this pattern, because repeated prefixes can reuse earlier work; see how prompt caching works.

Driver four: bigger prompts

Prompts keep growing. The a16z and OpenRouter study found the average prompt rose about fourfold, from roughly 1,500 tokens to over 6,000 per request, and the average completion nearly tripled, from about 150 to 400 tokens. Average sequence length rose from under 2,000 tokens in late 2023 to over 5,400 by late 2025. The same study found programming rose from about 11 percent of tokens in early 2025 to more than half in its final weeks, and coding work tends to carry whole files and long histories.

Context windows allow it. Anthropic's pricing page says Claude 4.6 and later models (except Haiku 5.5) include a 1 million token context window at standard pricing. More input means more prefill work, where the model reads the prompt, and more memory for the attention state. See tokens per second, TTFT and ITL.

Driver five: cheaper capability opens new uses

Falling prices might seem to cut demand, but the effect often runs the other way. When something gets cheaper, uses that were too expensive become viable. A company that would never have had a model read every support ticket, pull request or log line may do so once the price falls far enough.

Epoch AI estimated on 22 September 2026 that the cost of reaching a fixed benchmark score has fallen about 13 times a year since 2023. The details and caveats are in is AI getting cheaper.

Economists call the general pattern, where efficiency gains lead to more total use, the rebound effect, or the Jevons paradox in its strongest form. No source cited here tests it for AI, so treat it as a lens, not a finding. The usage figures above are consistent with it: Google's monthly tokens rose about 7 times in a year in which the cost of fixed capability fell by roughly an order of magnitude.

Driver six: AI inside other software

Models are becoming a component inside ordinary software. A code review tool calls one on every pull request. A help desk calls one on every ticket, again to draft a reply, and again to check it against policy. The person using the product may not think of it as AI, yet each action triggers model calls.

This ties request volume to business events, such as commits or tickets, which are large and steady. It also encourages chaining, where one call's output feeds the next. OpenRouter's press release of 26 May 2026 put its weekly volume at 25 trillion tokens, five times the level six months earlier, with more than 8 million users and over 400 models. Microsoft said the number of its Foundry customers at a 1 trillion token annualized rate grew 4 times in a year. Teams often discover this pattern late, when a feature that looked cheap in a demo meets production volume. A rule in front of a model can cut calls; see why LLM triage needs a deterministic gate.

What pushes the other way

Serving efficiency. The vLLM paper (September 2023) reported 2 to 4 times higher throughput than two earlier systems at the same latency, and the speculative decoding paper (November 2022) a 2 to 3 times speedup on one model with identical outputs.

Smaller and in house models. Microsoft said on 29 July 2026 it saw an 89 percent reduction of GPU costs in Dynamics 365 using its MAI-Voice-2-Flash model, and that MAI-Code-1-Flash achieved 10 percent lower median token usage in GitHub Copilot. These are company statements.

Adaptive thinking. Anthropic's documentation says newer models decide per request whether and how much to think, and at lower effort may skip thinking on easy inputs. In a tool loop, follow up requests that only process tool results can skip thinking.

Budgets and limits. Spend caps and cost per task targets cap demand; see setting LLM API spend limits.

Supply. Demand can only be served if capacity exists. Alphabet said in April 2026 it was compute constrained in the near term, and Microsoft said in July that demand continued to exceed available supply. Shortages can delay or reprice usage rather than remove it. For the capacity side, see OpenAI compute commitments explained and Anthropic compute commitments explained.

What is firm and what is uncertain

Firm:

• Company reported token volumes at Google, Microsoft and OpenRouter grew several times over in the year to mid 2026, by their own statements.
• Prompts and completions per request grew, and reasoning models took over half of tokens on OpenRouter by late 2025, per the a16z and OpenRouter study.
• Agents use several times more tokens than chat in Anthropic's published measurement, and thinking is billed as output on the major APIs.

Uncertain:

• The worldwide total. Company figures are self reported, defined differently and not audited.
• How representative OpenRouter is. It is one marketplace, and its own study found programming made up more than half of its tokens by late 2025.
• Whether agent token multiples from one 2025 research feature hold for other agents and newer models.
• How much efficiency, smaller models and supply limits will offset growth. The net can differ by company and product.

What to watch next

• Token disclosures on the next Alphabet and Microsoft earnings calls, and whether growth rates slow.
• Whether OpenAI or Anthropic publish token volumes, which they have not done on a regular basis.
• New published measurements of tokens per agent task, especially for long running agents.
• Capacity statements from hyperscalers, since supply limits can cap served demand.
• Epoch AI's next cost of capability update, to see whether falling prices keep pace with rising use.

What this means for a team that builds with AI

Plan for demand to grow faster than the first estimate. Estimate requests per user and tokens per request separately, and revisit both after a month of real traffic. Count hidden thinking and agent loops. Build a cost view per feature; see tracking AI spend per team. Treat unexplained spikes as a possible security signal, because stolen keys show up as token use; see the LLM bill spike runbook.

For teams that self host models, Safeguard's SBOM and dependency scanning covers the growing serving stack, with a free scan at https://gold.safeguard.sh/scan.

Quick check

Sources

• Google, I/O 2026: Welcome to the agentic Gemini era (19 May 2026)
• Alphabet, Q1 2026 earnings call (29 April 2026)
• Microsoft, FY26 fourth quarter earnings call transcript (29 July 2026)
• TechCrunch, ChatGPT reaches 900M weekly active users (27 February 2026)
• OpenRouter press release via Business Wire, republished by 01net (26 May 2026)
• a16z and OpenRouter, State of AI: an empirical 100 trillion token study (December 2025)
• Anthropic, How we built our multi-agent research system (13 June 2025)
• Anthropic, Steering thinking documentation
• Anthropic, Extended thinking documentation
• Anthropic, Pricing documentation (accessed 19 October 2026)
• Google, Gemini Developer API pricing (updated 16 October 2026)
• Epoch AI, The plunging price of thought (22 September 2026)
• Snell et al., Scaling LLM Test-Time Compute Optimally, arXiv 2408.03314
• Stanford HAI, 2025 AI Index Report, Technical Performance
• Wu et al., Sustainable AI, arXiv 2111.00364
• Kwon et al., PagedAttention, arXiv 2309.06180
• Leviathan et al., Speculative Decoding, arXiv 2211.17192

Last reviewed: 19 October 2026.