Is AI getting cheaper? Cost per token trends and caveats

Yes, for a fixed level of ability. Epoch AI estimated in September 2026 that the cost of reaching a given benchmark score has fallen about 13 times a year since 2023. No, if you mean the newest models or your total bill. Frontier list prices have held or risen, reasoning uses more tokens per task, and usage keeps growing.

Key numbers at a glance

• About 13x per year: fall in the cost of reaching a fixed benchmark score since 2023, roughly 47 percent per quarter (Epoch AI analysis, 22 September 2026).
• About 75x per year, then about 4.7x: the decline right after a capability level first appears, and two years later (Epoch AI, 22 September 2026).
• 725 fold: drop in cost per question to match o3's GPQA Diamond score, from about 0.30 dollars in January 2025 to 0.0004 dollars, in under 18 months (Epoch AI, 22 September 2026).
• 9x to 900x per year, median 50x: the earlier per token estimate across six benchmarks; 200x median using only 2024 and later data (Epoch AI, 12 March 2025).
• 5x to 10x per year: price decline for a given benchmark score on frontier models; the price of running frontier models through benchmarks rose 3x to 18x per year (Gundlach and co-authors, arXiv paper revised 23 March 2026).
• Over 280 fold: fall in the cost of GPT-3.5 level performance, November 2022 to October 2024 (Stanford AI Index 2025).
• 15 to 4 dollars: Anthropic's input price per million tokens for its Opus tier, from Claude Opus 4 to Claude Opus 5.5; output went from 75 to 20 dollars (Anthropic pricing page, as of 19 October 2026).
• About 30 percent more tokens: for the same text with the tokenizer used by Claude 4.7 and later models (Anthropic pricing page, as of 19 October 2026).
• Doubling on 1 December 2026: Gemini 4 Argon list price moves from 2 to 4 dollars input and 10 to 20 dollars output per million tokens (Google Gemini API pricing page, updated 16 October 2026).
• About 1,500 to over 6,000: average prompt tokens per request on OpenRouter, with completions rising from about 150 to 400 tokens (a16z and OpenRouter study, December 2025).

Price per token is a price, not a cost

AI providers that sell model access through an API charge by the token. A token is a small chunk of text, often a few characters or part of a word. Input tokens (what you send) and output tokens (what the model writes back) carry separate rates, quoted per million tokens. Output usually costs several times more than input.

Price per token is the easiest number to track, so most trend studies use it. It is still a price. It is what a customer pays, not what serving the request costs the provider. The provider's cost depends on chips, electricity, how busy the hardware is and the serving software, and providers do not publish those figures. This post makes no claim about any provider's internal costs or margins. When the studies below say prices fell, they mean market prices fell.

There is a second distinction that matters even more. A price per token says nothing about how many tokens a task needs. Reasoning models think before they answer and are billed for that thinking. Agents make many calls per task. So two numbers can move in opposite directions at the same time: price per token at a fixed quality level, and spend per completed task.

The newest measurement: Epoch AI, September 2026

Epoch AI is an independent research group. On 22 September 2026 it published "The plunging price of thought" by Luke Emberson and David Roodman. The study asks a careful question: what is the cheapest way, at each point in time, to reach a given score on a benchmark, counting all the tokens a model actually uses, including reasoning tokens?

That design fixes a weakness in older studies. Earlier work compared per token prices and left reasoning models out, because their long hidden reasoning makes per token comparisons misleading. The September study instead measures the cost of getting the score, using benchmark transcripts to estimate token budgets.

The headline result is a decline of about 47 percent per quarter, or roughly 13 times per year, since 2023. The rate differs by task. Game puzzles fell 39 to 43 percent per quarter (about 7 to 10 times a year). Math problems fell 50 to 52 percent per quarter (about 16 to 19 times a year).

The study also found that the speed of the decline depends on how new the capability is. In the quarter when a performance level is first reached, cost falls about 66 percent per quarter, which Epoch converts to roughly 75 times a year. Two years later, the rate has slowed to about 32 percent per quarter, or about 4.7 times a year. In plain terms, a new capability gets cheap quickly at first, then more slowly.

Epoch's worked example is striking. OpenAI's o3, released on 31 January 2025, scored about 75 percent on GPQA Diamond, a set of hard science questions, at roughly 0.30 dollars per question. A later model, which Epoch names GPT-5.6 Luna, matched that score at 0.0004 dollars per question. That is a 725 fold drop in under 18 months.

For context, the authors compare the rate with other technologies. US residential electricity prices fell about 1.05 times a year from 1892 to 1973. Compute fell about 1.51 times a year from 1940 to 2001. DNA sequencing fell about 1.84 times a year from 2001 to 2025. On Epoch's measure, LLM inference is falling faster than all of them.

What the earlier studies found

The September study builds on Epoch's March 2025 data insight by Ben Cottier and co-authors, which looked only at per token prices for non reasoning models. That study found price declines between 9 times and 900 times a year depending on the benchmark and threshold, with a median of 50 times. When the authors dropped data from before 2024, the median rose to 200 times a year. They warned that the fastest rates came from the most recent year and might not persist. Prices in that study were a 3 to 1 weighted average of input and output rates.

Its tables give two clean data points. GPT-4 (the March 2023 snapshot) was listed at 37.50 dollars per million blended tokens. By February 2025, Gemini 2.0 Flash was listed at 0.18 dollars and scored higher on MMLU, a broad knowledge test.

Stanford's 2025 AI Index reached a similar conclusion from a different angle. It reported that the cost of running a system at GPT-3.5 level fell by more than 280 times between November 2022 and October 2024. The same report said hardware costs had declined by 30 percent a year and energy efficiency had improved by 40 percent a year. It also noted a size effect: Microsoft's Phi-3-mini, with 3.8 billion parameters, reached a score on MMLU that in 2022 had required a 540 billion parameter model, a 142 fold reduction in parameters.

A third view comes from Hans Gundlach and co-authors at MIT, in a paper first posted in November 2025 and revised on 23 March 2026. Using Artificial Analysis and Epoch data, they found that the price of a given benchmark score on frontier models fell about 5 to 10 times a year. After removing hardware price declines and looking at open models, they estimated algorithmic efficiency gains of about 3 times a year. They also found that the price of running frontier models through benchmarks is rising, by about 3 to 18 times a year, because frontier models are larger and reason more.

These numbers disagree on the exact rate (13x, 5x to 10x, a median of 50x) because they measure different things over different windows. They agree on direction: a fixed level of ability gets much cheaper every year.

Why frontier list prices do not follow the trend

The trend is about old capability levels getting cheap. The price of the newest, strongest model is a separate question, and the current price pages show it moving in both directions.

Anthropic's pricing page, as listed on 19 October 2026, shows the Opus tier falling over time. Claude Opus 4 and Opus 4.1 are listed at 15 dollars per million input tokens and 75 dollars per million output tokens. Opus 4.5 through Opus 5 are listed at 5 and 25 dollars. Claude Opus 5.5 is listed at 4 and 20 dollars. The small model end is far cheaper: Claude Haiku 5.5 is listed at 0.10 and 0.50 dollars for prompts up to 100,000 tokens. At the same time, the page lists a higher tier above Opus, Claude Fable 5 and 5.1, at 10 and 50 dollars.

Google's Gemini API pricing page, updated on 16 October 2026, shows scheduled increases. Gemini 4 Argon is listed at 2 dollars input and 10 dollars output per million tokens through 30 November 2026, then 4 and 20 dollars from 1 December 2026. Gemini 3.8 Flash is listed at 0.75 and 3.75 dollars through 31 December 2026, then 1.50 and 7.50 dollars from 1 January 2027. The page states that its output prices include thinking tokens.

Capacity is one plausible factor, though no source here ties these specific prices to it. On Microsoft's earnings call of 29 July 2026, chief financial officer Amy Hood said demand continued to exceed available supply and pointed to pricing in the spot market for compute. Alphabet's chief executive Sundar Pichai said on 29 April 2026 that Google was compute constrained in the near term. When supply is tight, a provider has less reason to cut the price of its newest model.

So "AI is getting cheaper" is true for a capability that has been around for a while. It is not a promise that the next flagship will cost less than the last one.

Why your bill can still rise

Tokens per task are growing. The a16z and OpenRouter study of more than 100 trillion tokens, published in December 2025, found that the average prompt grew from about 1,500 tokens to over 6,000 per request, and the average completion from about 150 to 400 tokens, over the study window. It also found that reasoning models went from a negligible share of tokens in early 2025 to more than half.

Reasoning is billed as output. Anthropic's documentation says thinking tokens are billed as output tokens and that the billed output count does not match the visible text, because you pay for the full thinking process. Google's price page likewise includes thinking tokens in the output price. Stanford's 2025 AI Index gave an early example: o1 was nearly six times more expensive and 30 times slower than GPT-4o.

Tokenizers change. A tokenizer splits text into tokens, and different model generations split it differently. Anthropic's pricing page says the tokenizer used by Claude 4.7 and later models produces approximately 30 percent more tokens for the same text, with the exact increase depending on content. A lower rate per token can be partly offset by more tokens. See same price per token, same bill? A myth.

You may move up a tier. The falling price applies to a fixed capability. Teams that switch to a stronger model for harder work pay that model's price.

Usage grows. Cheaper capability makes new uses worth building. The companion post on why AI inference demand keeps growing covers the usage side with 2026 figures.

The useful unit for a business is cost per completed task, covered in cost per task, not cost per token.

Where the savings show up

Large buyers do report savings from moving work to smaller or in house models. These are company statements, not audited figures. On 29 July 2026, Microsoft's chief executive Satya Nadella said the company saw an 89 percent reduction of GPU costs in Dynamics 365 with its MAI-Voice-2-Flash model, and up to 84 percent lower GPU costs in PowerPoint with MAI-Image-2.5. At Google I/O on 19 May 2026, Pichai claimed that companies processing about 1 trillion tokens a day could save over 1 billion dollars a year by moving 80 percent of their workloads to Gemini 3.5 Flash.

The pattern in both statements is the same as in the Epoch data. The savings come from running a task on a cheaper model that is now good enough, not from the flagship getting cheaper.

Reading a price page without being fooled

Compute your own blend. Epoch's 3 to 1 blend of input and output is a general average. A summarizer reads a lot and writes a little; a drafting tool does the reverse. Use your real traffic.

Check the modifiers. Anthropic's page lists a 50 percent Batch API discount, cache reads at a fraction of the base input price, a 1.1 times multiplier for US only inference on Claude 4.6 and later, and fast mode at a premium. Google's page lists a 50 percent batch discount and dated price changes. These can matter more than the headline rate.

Count invisible tokens. A model with a low output rate can cost more than a pricier one if it thinks at length.

Compare at the same quality bar. Copy Epoch's method on a small scale: set a pass mark for your task, run candidates on real inputs, and compare the cost of the ones that pass.

Date every number. The figures here are snapshots as of 19 October 2026. Re-read provider pages before you commit.

What is firm and what is uncertain

Firm:

• The cost of a fixed level of measured ability has fallen by roughly an order of magnitude or more per year since 2023. Three independent studies (Epoch twice, Gundlach and co-authors) and the Stanford AI Index agree on direction.
• The decline is fastest just after a capability first appears and slows after that (Epoch, September 2026).
• Thinking tokens are billed as output on the major APIs, per Anthropic's and Google's own pricing documentation.
• Listed prices for some current models are scheduled to rise, per Google's pricing page updated 16 October 2026.

Uncertain:

• The exact annual rate. Estimates range from 5x to 13x for the newer methods and up to a 200x median in the 2025 per token study, depending on method and window.
• Whether the rate will persist. Epoch notes possible tuning to benchmarks, noisy data covering only about three years, and unreported standard errors.
• Why prices fall. Smaller models and cheaper hardware are named causes. Epoch found no clear evidence on margins, and no source here establishes any provider's internal cost.
• Why some list prices are rising. Tight compute supply is a plausible factor, but the price pages give no reason.

What to watch next

• Whether Google's scheduled Gemini increases take effect on 1 December 2026 and 1 January 2027 as listed.
• Epoch's next update to its cost of capability series, and whether the post SOTA slowdown holds.
• Supply commentary on the next round of hyperscaler earnings calls, since capacity tightness affects frontier pricing.
• Whether tokenizer changes spread to other providers, which would change bills at unchanged per token rates.
• New published data on tokens per task for agents and reasoning models.

What it means for planning

Plan with a conservative rate. Re-test cheaper models every quarter against your own pass mark, because the capability you need today is likely to get cheaper. Budget in tasks, including thinking tokens, and set spend limits so a loop or spike cannot surprise you; see setting LLM API spend limits. For the wider economics behind these prices, see Anthropic revenue, profit and gross margin, reported vs estimated and OpenAI compute commitments explained.

Teams that move work to self hosted open models take on the serving stack's dependencies; Safeguard's SBOM and dependency scanning covers that stack, with a free scan at https://gold.safeguard.sh/scan.

Quick check

Sources

• Epoch AI, The plunging price of thought (22 September 2026)
• Epoch AI, LLM inference prices have fallen rapidly but unequally across tasks (12 March 2025)
• Gundlach, Lynch, Mertens and Thompson, The Price of Progress: Price Performance and the Future of AI, arXiv 2511.23455 (revised 23 March 2026)
• Stanford HAI, 2025 AI Index Report
• Stanford HAI, 2025 AI Index Report, Technical Performance
• Anthropic, Pricing documentation (accessed 19 October 2026)
• Anthropic, Steering thinking, pricing section
• Google, Gemini Developer API pricing (updated 16 October 2026)
• a16z and OpenRouter, State of AI: an empirical 100 trillion token study (December 2025)
• Microsoft, FY26 fourth quarter earnings call transcript (29 July 2026)
• Alphabet, Q1 2026 earnings call (29 April 2026)
• Google, I/O 2026: Welcome to the agentic Gemini era (19 May 2026)

Last reviewed: 19 October 2026.