Training vs inference: how AI compute is actually split

No one publishes a reliable industry wide split between training and inference compute. The disclosed and estimated figures point in one direction: inference is taking a growing share. Meta reported a rough 70 percent inference share of power capacity in 2021, and Deloitte predicted two thirds of AI compute for 2026, but OpenAI's 2024 spending ran the other way.

Key numbers at a glance

• 10:20:70: Meta's rough power capacity split across experimentation, training and inference (Wu and co-authors, paper posted October 2021).
• About 7 billion dollars: OpenAI's estimated 2024 cloud compute spend, about 5 billion for research and development (R&D) and 1.8 billion for inference (Epoch AI estimate from reported investor documents, published 10 October 2025).
• About 469 million dollars, roughly 7 percent: Epoch's estimate for the final training runs of OpenAI's released 2024 models, including about 386 million for GPT-4.5 with a 90 percent range of 170 to 890 million (Epoch AI, 10 October 2025).
• 9.6, 22.6 and 12.3 percent: final training runs as a share of R&D compute spending at OpenAI, MiniMax and Z.ai (Epoch AI, 23 March 2026, from IPO filings and reports).
• Two thirds: Deloitte's prediction for inference's share of all AI compute in 2026 (Deloitte TMT Predictions, 18 November 2025; a projection).
• Over 50 billion dollars: Deloitte's predicted 2026 market for inference optimized chips (18 November 2025; a projection).
• 10 to 15 percent: OpenAI's estimated share of world operational AI compute at the end of 2025, about 1.7 million H100 equivalents (Epoch AI analyst estimate, 20 May 2026).
• About 25 percent, up to 50: how much hyperscaler hosted inference of OpenAI models may add to OpenAI's own compute total (Epoch AI, 20 May 2026).
• Over 3.2 quadrillion tokens a month: processed across Google's surfaces, up about 7 times in a year (Google, 19 May 2026; company statement).
• 4x: year on year growth in the number of Microsoft Foundry customers at a 1 trillion token annualized run rate (Microsoft earnings call, 29 July 2026).

Three buckets, not two

Training is the work of producing a model. Enormous numbers of calculations adjust the model's internal numbers, called parameters, until it predicts well. A training run starts, runs for weeks or months on thousands of accelerators, and ends with a finished model.

Inference is what happens when someone uses the model. A request comes in, the model reads it and writes an answer one token (a small chunk of text) at a time. Each request is a small job, and the jobs keep arriving for as long as people use the product.

A third bucket is easy to miss. Before a lab trains a final model, it runs many smaller experiments: testing data mixes, trying architectures, running attempts that fail. This is research or experimentation compute. The data below shows it can be much larger than the final training runs. Whenever you see a headline split, check whether experimentation is counted as training, counted separately or left out.

The shapes differ too. Training is a large, lumpy expense per model version. Inference is a running cost that grows with users and with how much each user asks. A model nobody uses costs a lot to build and little to serve. A model used by hundreds of millions of people can cost far more to serve over its life than to build.

What Meta reported

The clearest public breakdown from a large operator is still a 2021 paper by Carole-Jean Wu and 24 co-authors at Meta (then Facebook), "Sustainable AI: Environmental Implications, Challenges and Opportunities." It states that the company observed a rough power capacity breakdown of 10:20:70 across experimentation, training and inference.

The paper adds growth figures from that period. AI use cases drove a 2.9 times increase in training infrastructure capacity over a year and a half. The company saw trillions of inferences per day across its data centers, and a 2.5 times increase in inference infrastructure capacity.

Limits apply. The figures describe one company, mostly ranking and recommendation models that run on every feed view for billions of people. That is an unusually inference heavy pattern. The paper predates large chat models. It measures power capacity, not money. The authors call the split rough.

The paper also shows the answer varies by model even inside one company. For recommendation use cases it found the carbon footprint split evenly between training and inference. For a language model it studied, inference dominated, at 65 percent against 35 percent for training.

What Epoch AI estimated for OpenAI

The second public picture comes from Epoch AI, an independent research group. In a data insight by Josh You published on 10 October 2025, Epoch estimated OpenAI's 2024 cloud compute at about 7 billion dollars. About 5 billion went to R&D and 1.8 billion to inference.

Inside R&D, Epoch's estimates are revealing. The final training run of GPT-4.5 cost about 386 million dollars, with a 90 percent range of 170 to 890 million. Other released models, including GPT-4o and post training for the o series, added about 83 million. That leaves about 4.5 billion for basic research, derisking runs and unreleased models. Final training runs of released models came to about 469 million dollars, roughly 7 percent of total compute spend.

On those figures inference was roughly a quarter of the total, the opposite of Meta's pattern. The reason is timing. In 2024 OpenAI was spending heavily to build its next models, while its user base was still growing.

Epoch's caveats matter. The numbers come from September 2024 investor documents reported by The Information and The New York Times, and partly rely on projections for the fourth quarter. Research compute was reported as amortized over several years, and Epoch assumed two years. The inference figure excludes Microsoft's own costs for running OpenAI models inside Microsoft products. These are rental costs, not data center capital costs.

New 2026 data: final training runs are a minority of R&D

A March 2026 Epoch analysis by Jean-Stanislas Denain and Cheryl Wu extended the comparison using figures from the Hong Kong IPO documents of two Chinese labs. Final training runs made up 9.6 percent of R&D compute spending at OpenAI (2024), 22.6 percent at MiniMax and 12.3 percent at Z.ai. The MiniMax and Z.ai windows do not cover full calendar 2025, and the Z.ai figure captures third party service fees rather than owned hardware.

The authors add a point that changes how to read every dollar split. R&D workloads usually run at lower utilization (model FLOPs utilization, or MFU, the share of a chip's peak math the work actually uses) than final training runs. So the same spending on R&D may buy fewer actual calculations. A split by dollars and a split by operations can differ.

What the 2026 estimates say about scale

In a May 2026 newsletter piece, Epoch's Josh You estimated that OpenAI had about 1.7 million H100 equivalents (a common unit that converts different chips to the compute of Nvidia's H100) at the end of 2025, up from about 400,000 in 2024. That would be about 10 to 15 percent of world operational AI compute. Anthropic probably had over 1 million. The world total was about 20 million H100 equivalents sold, with about 16 million deployed assuming a one quarter lag. The author describes these as more tentative than Epoch's standard data work.

The same piece shows why lab level splits undercount inference. Inference of OpenAI models on hyperscaler products such as Microsoft's Azure API may add around 25 percent to OpenAI's total, with 50 percent as an upper bound, and third party cloud inference adds a similar 25 percent for Anthropic. Those workloads are inference by definition but do not appear in the lab's own spending.

Deloitte offered an industry view in its 2026 Technology, Media and Telecommunications Predictions, released on 18 November 2025. It predicted that inference would make up two thirds of all AI compute in 2026, that the market for inference optimized chips would exceed 50 billion dollars, and that combined training and inference chips in data centers would keep dominating a 200 billion dollar AI chip market. These are predictions, not measurements, and the release does not publish the underlying model.

Demand signals from the operators

No large cloud company publishes a training versus inference split for its fleet. Several disclose usage figures that show inference volume growing fast. These are company statements, not audited data.

• Google said on 19 May 2026 that it processes over 3.2 quadrillion tokens a month across its surfaces, up about 7 times from roughly 480 trillion a year earlier and from 9.7 trillion two years earlier.
• Alphabet said on 29 April 2026 that 330 Google Cloud customers had each processed more than 1 trillion tokens in the past 12 months, and that Google was compute constrained in the near term.
• OpenAI said on 27 February 2026 that ChatGPT had reached 900 million weekly active users, up from 800 million in October 2025, as reported by TechCrunch. Every one of those users generates inference, not training, demand.
• Microsoft said on 29 July 2026 that the number of Foundry customers at a 1 trillion token annualized run rate had increased 4 times year on year. Its chief financial officer, Amy Hood, said demand continued to exceed available supply, and described the Azure infrastructure as "pretty fungible" in the context of serving many different models.

Hood was talking about serving, not about training. Still, where accelerators can be moved between workloads, a fixed split at one moment says less than the trend in how capacity is used.

Why the split moves

Usage pushes inference up. Every new user and feature adds requests, while a training run is paid once per model. The token figures above show request volume rising quickly.

Longer outputs push inference up. Reasoning models think before answering and agents take many steps, so each task uses more tokens. The a16z and OpenRouter study of over 100 trillion tokens, published in December 2025, found reasoning models went from a negligible share of tokens in early 2025 to more than half. See why AI inference demand keeps growing.

Bigger frontier runs and research push training up. Each new generation tends to need more compute, and Epoch's data shows experimentation is the largest single slice of R&D.

Efficiency pushes the inference share down. Batching, caching, quantization and speculative decoding let the same hardware serve more requests. See continuous batching and PagedAttention.

The two trade against each other. A paper by Charlie Snell and co-authors (August 2024) found that, in a comparison matched on total operations, a smaller model given extra compute at answer time could outperform a model 14 times larger on problems where the smaller one already had a non trivial success rate. Labs can choose between a bigger training run and more thinking at inference, so they increasingly plan the two as one budget.

Why the hardware side cares

Training and inference stress hardware differently. Training moves huge amounts of data between many chips for weeks and rewards raw compute and fast interconnect. Inference is many small jobs where memory capacity and memory bandwidth often limit how fast tokens come out, and where cost per token and latency matter most. That is why chip makers and clouds increasingly market separate training and inference products; see AI chip market share in 2026. It also shapes where capacity is built and how labs sign contracts; see OpenAI compute commitments explained and Anthropic compute commitments explained.

How to read a claim about the split

1. Whose compute? One company, one cloud, or the industry? Industry totals are estimates.
2. What counts as training? Final runs only, or experiments and failed runs too?
3. What unit? Dollars, power capacity, chip hours or operations can give different ratios.
4. What date? A 2021 social media fleet and a 2024 lab in a build phase describe different worlds.
5. Who says so? A company paper, a journalist's reading of investor documents, an analyst model or a prediction?

What is firm and what is uncertain

Firm:

• Meta's 2021 paper reported a rough 10:20:70 power capacity split for its own fleet, dominated by inference.
• Final training runs are a minority of R&D compute spending at the three labs Epoch could measure (9.6 to 22.6 percent).
• Inference volume at Google and Microsoft grew several times over in the past year, by their own statements.

Uncertain:

• The industry wide split today. Deloitte's two thirds figure is a prediction, and no operator publishes a fleet split.
• OpenAI's split after 2024. Epoch's dollar estimate is for 2024, from reported investor documents with partial projections.
• How much inference sits outside lab accounts, on hyperscaler products. Epoch's 25 percent adjustment is an estimate with a 50 percent upper bound.
• Whether dollar splits and operation splits agree, given lower utilization in research work.

What to watch next

• Any audited filing from an AI lab that separates training, research and inference compute, as the MiniMax and Z.ai IPO documents partly did.
• Epoch's updates to its AI chip owner and compute spend datasets for 2026.
• Hyperscaler token disclosures on the next earnings calls, which serve as a proxy for inference volume.
• Whether Deloitte's 2026 prediction is revisited with measured data in its next report.

For teams that run their own inference servers, the practical point is that inference is the part most organizations operate. Safeguard's SBOM and dependency scanning covers inference container images and their packages, with a free scan at https://gold.safeguard.sh/scan.

Quick check

Sources

• Wu et al., Sustainable AI: Environmental Implications, Challenges and Opportunities, arXiv 2111.00364
• Epoch AI, Most of OpenAI's 2024 compute went to experiments (10 October 2025)
• Epoch AI, Final training runs account for a minority of R&D compute spending (23 March 2026)
• Epoch AI, Frontier labs don't use most AI compute (yet) (20 May 2026)
• Deloitte, 2026 TMT Predictions press release (18 November 2025)
• Google, I/O 2026: Welcome to the agentic Gemini era (19 May 2026)
• Alphabet, Q1 2026 earnings call (29 April 2026)
• TechCrunch, ChatGPT reaches 900M weekly active users (27 February 2026)
• Microsoft, FY26 fourth quarter earnings call transcript (29 July 2026)
• a16z and OpenRouter, State of AI: an empirical 100 trillion token study (December 2025)
• Snell et al., Scaling LLM Test-Time Compute Optimally, arXiv 2408.03314

Last reviewed: 19 October 2026.