AI Accelerator TCO: Buy vs Rent, Capex, Power, Cooling and Utilization

Buying AI accelerators beats renting them on demand once you keep them busy enough, and for a typical 8-GPU H100 server that threshold is lower than most people expect: roughly 20% to 40% average utilization against public on-demand prices, depending on what you pay and how you finance it. Against a multi-year reservation the answer flips, because both options charge you for idle hours.

This guide builds that answer from parts you can check. It sets out every cost of owning an accelerator (capital, amortization, power, PUE and cooling, facility space, networking, maintenance and staff), sets each one as a cited fact or a stated assumption, and compares the result with public cloud prices for H100, H200, B200, GB200, MI300X, MI355X, TPU and Trainium capacity that we checked on 25 October 2026. Every number below came out of a Python calculation, and the full calculator is included so you can replace our assumptions with your own quotes.

One rule shapes the whole post. NVIDIA, AMD, Google and AWS do not publish purchase prices for their data center accelerators, so we never state one. Instead we treat capex as the unknown and solve for it: the maximum all-in price per accelerator at which buying still wins. That number is more useful in a negotiation than any rumoured street price.

Key takeaways

• Owning converts a variable cost into a fixed one. An owned accelerator costs the same per hour whether it is busy or idle, so the cost per useful hour is the fixed hourly cost divided by utilization. Renting on demand charges only for the hours you hold.
• Break-even utilization is a ratio. It equals the owned fixed cost per hour divided by the on-demand price. With an illustrative all-in capex of $30,000 per H100, a four-year life and an 8% cost of capital, the fixed cost is about $1.72 per GPU-hour and break-even against the AWS on-demand price of $6.88 is 25% utilization.
• Capital dominates. In that example, capital recovery and maintenance are 72% of the hourly cost; colocation is about 18%; electricity is only about 6%. Moving electricity from 5 to 20 cents per kWh changes the total by about 10%.
• Reservations change the comparison. Azure's three-year reservation for ND H100 v5 works out to $5.40 per GPU-hour, 56% below its on-demand rate, and Google's three-year TPU commitments are 55% below on-demand. Against a reservation you pay for idle hours either way, so utilization cancels and the question becomes whether you can run the hardware more cheaply than the provider.
• Training and inference fleets behave differently. A training cluster with a queue of runs stays above 80% busy; Meta reported more than 90% effective training time for Llama 3. An inference fleet follows daily demand, so the cheapest design often owns the base load and rents the peak.
• The generation treadmill is a real cost. On Oracle's price list, GB200 capacity rents for $6.40 per peak dense BF16 PFLOP-hour versus $10.11 for H100, 37% less. Whatever you buy competes with cheaper rented compute within its own useful life.
• The hardware is only part of the stack. Drivers, CUDA or ROCm, NCCL, serving frameworks and container images ship on every node you own, and they need an SBOM and a patch cadence just as much as the rest of your software.

The whole question in one formula

Everything in this post reduces to one comparison. For an accelerator you own, the cost per hour of useful work is:

owned cost per useful hour = F / u

  F = fixed cost per accelerator-hour (capital + maintenance + energy + facility + staff)
  u = utilization, the fraction of all hours the accelerator does useful work

For an accelerator you rent on demand, the cost per useful hour is simply the hourly price p, because you release the capacity when you do not need it. Owning wins when F / u is less than p, which rearranges to:

break-even utilization  u* = F / p

If you rent with a commitment (a one-year or three-year reservation, a committed use discount, or a capacity block you must pay for whether you use it or not), you also pay for idle hours. Then the comparison is F against the committed price q, and utilization drops out of the comparison entirely.

The rest of the post is about estimating F honestly, finding p and q on public price pages, and understanding what moves u.

              OWN                                 RENT ON DEMAND
  +--------------------------------+       +--------------------------------+
  | capex  -> amortize over n yrs  |       |                                |
  | + maintenance and support      |       |   price p per accelerator-hour |
  | + energy  (kW x PUE x $/kWh)   |       |   paid only while you hold it  |
  | + facility (space, cooling,    |       |                                |
  |   power delivery)              |       +--------------------------------+
  | + staff                        |                      |
  +--------------------------------+                      |
                 |                                        |
          F per hour, all hours                   p per hour held
                 |                                        |
                 v                                        v
         cost per useful hour = F/u        cost per useful hour = p
                 \                                        /
                  \_____ owning wins when u > F / p _____/

What it costs to own an accelerator

Each component below is either a fact from a primary source or an assumption we state and then sweep. The table gives the base case used in the worked examples.

Capex: what is in it, and why we do not quote a price

The capex of an accelerator in a cluster is much more than the accelerator. A complete line item usually includes the server or rack (CPUs, host memory, NVMe, power supplies, the baseboard with the accelerators and their scale-up switch), the share of the scale-out network (network adapters, leaf and spine switches, optics and cables), the share of storage, installation and burn-in, and spares.

NVIDIA's DGX H100 user guide shows how much sits around the GPUs in one 8-GPU node: two Intel Xeon 8480C CPUs, four fourth-generation NVLink switches providing 900 GB/s of GPU-to-GPU bandwidth, and six 3.3 kW power supplies. The DGX B200 guide lists two Xeon 8570 CPUs and two fifth-generation NVLink switches with 14.4 TB/s of aggregate bandwidth. All of that is capex before the first rack switch.

None of NVIDIA, AMD, Google or AWS publishes a purchase price for these systems. Prices you see quoted in articles are reported street prices, not primary sources, so we do not use them. Instead this post does two things:

1. It solves for the break-even capex: the most you could pay per accelerator, all in, and still beat a given cloud price at a given utilization.
2. Where a capex number is needed to show a curve, it uses a sweep grid ($20,000, $30,000, $40,000 and $50,000 per H100 slot; $40,000, $60,000 and $80,000 per B200 slot). These are grid points for arithmetic, not estimates of market prices. Replace them with your quote.

Amortization: three, four or five years

How long you spread the capex over is the biggest single assumption in the model, and it is partly an accounting choice and partly a bet on obsolescence.

Large cloud providers disclose their choices. In its 2024 annual report on Form 10-K, Amazon said it completed a useful life study of servers and networking equipment in the fourth quarter of 2024 and changed "a subset of our servers and networking equipment" from six years to five years, effective 1 January 2025, expecting a reduction of about $0.7 billion in 2025 operating income from that change alone. A provider with its own data centers and a deep resale and reuse pipeline still judged six years too long for part of its fleet.

For a buyer without that reuse pipeline, three to five years is the realistic window. We use the capital recovery factor, which turns a capex into an equal annual payment that repays it with interest:

CRF(r, n) = r (1 + r)^n / ((1 + r)^n - 1)

At 8%, the annual payment per dollar of capex is:

Per accelerator-hour, every $10,000 of capex costs:

That table is the fastest sanity check in this whole subject. If your all-in quote is $35,000 per accelerator and you plan a four-year life, you are paying about 3.5 x $0.413, or $1.45 per accelerator-hour, before a single kilowatt-hour of electricity.

Power: rated, maximum and operational

A server's rated power is the absolute maximum its power supplies are designed to deliver. NVIDIA's DGX H100/H200 user guide gives 10.2 kW maximum for the system, fed by six 3.3 kW supplies in a 4+2 redundant arrangement. The DGX B200 guide gives 14.3 kW maximum, with six 3.3 kW supplies in 5+1 redundancy. The H100 SXM module itself has a maximum thermal design power of up to 700 W according to NVIDIA's product page; AMD lists a typical board power of 750 W for the MI300X and 1,400 W for the MI355X.

Rated power is what you size electrical and cooling infrastructure for, but it is not what you pay for each hour. Lawrence Berkeley National Laboratory's 2024 United States Data Center Energy Usage Report makes this distinction explicitly. It defines rated, maximum, operational and idle power, and it notes that servers "rarely, if ever" draw their full rated wattage. For AI servers it models operational draw at 70% of rated, based on a measured analysis in which 8-GPU H100 nodes averaged 74% of rated power across single-node and multi-node training workloads, and it varies the figure between 60% and 80% for 2024 to 2028.

We use 70% of rated for every hour the node is powered. That slightly overstates energy for a node that sits idle part of the time, which keeps the estimate conservative for owning.

For one 8 x H100 node at 70% of 10.2 kW and a PUE of 1.2, that is:

10.2 kW x 0.70 x 1.2 x 8,760 h = 75,056 kWh a year at the facility meter
75,056 kWh x $0.0964 = $7,235 a year, or about $0.103 per GPU-hour

The same arithmetic for an 8 x B200 node at 14.3 kW gives 105,225 kWh and $10,144 a year, about $0.145 per GPU-hour.

PUE and cooling: air versus liquid

Power usage effectiveness (PUE) is total facility energy divided by IT equipment energy. A PUE of 1.2 means 0.2 kWh of cooling, power conversion and lighting for every kWh the servers consume.

Three primary data points bracket the realistic range:

• Google reports a trailing twelve-month PUE of 1.09 across its large-scale data center fleet in 2025, including all sources of overhead.
• Google's same page cites the Uptime Institute's 2025 Global Data Center Survey, in which the average PUE of respondents' data centers was 1.54.
• LBNL's 2024 report models the US average falling from 1.6 in 2014 to 1.4 in 2023, with hyperscale and colocation facilities below 1.4, and projects 1.15 to 1.35 by 2028. It also cautions that simulated PUEs assume systems operate as designed, which is "rarely the case".

Cooling choice enters a TCO model in two places. The first is PUE: LBNL's simulations show PUE varying by cooling system type, with economizers, adiabatic cooling and dry coolers lowering it. The second is facility capex and density, which matters more for current hardware.

The two Blackwell systems in this post sit on either side of the air and liquid line:

• DGX B200 is air cooled. Its user guide lists 1,550 CFM of airflow and a heat output of 48,794 BTU per hour, which is the 14.3 kW rated power converted to BTU (14.3 x 3,412 = 48,794). Moving that much air through a 10U chassis is why many existing halls can hold only a few such nodes per rack.
• GB200 NVL72 is liquid cooled. NVIDIA describes it as a rack-scale, liquid-cooled design connecting 36 Grace CPUs and 72 Blackwell GPUs in one NVLink domain, and says liquid cooling increases compute density and reduces floor space. NVIDIA describes the GB300 NVL72 as fully liquid cooled as well.

In this model, cooling shows up through PUE (swept from 1.09 to 1.54) and through the facility charge per kW. A liquid-cooled build typically trades higher facility capex per kW for lower PUE and far higher density per rack. We do not have a primary source for the capex of a liquid cooling retrofit, so we leave it inside the facility assumption and show how much that assumption matters.

Electricity price

The US Energy Information Administration publishes average retail prices by sector and state every month. In Table 5.6.A of its Electric Power Monthly, the preliminary August 2026 averages were:

We use 9.64 cents as the base case and sweep from 5 to 20 cents, which covers a long-term power purchase agreement at the low end and an expensive commercial tariff at the high end.

Facility, networking, maintenance and staff

These are the costs that are hardest to source publicly, and each is a stated assumption.

• Facility. Whether you lease colocation or build, you pay for provisioned power capacity, cooling capacity and space. We express it as $175 per rated kW per month and sweep $100 to $250. For one 8 x H100 node that is $21,420 a year, or $0.306 per GPU-hour; at the ends of the sweep it is $0.175 and $0.437.
• Networking. We fold the scale-out network into the all-in capex per accelerator, because it is bought with the cluster and depreciated with it. If your quote lists the network separately, add its per-accelerator share to the capex you test.
• Maintenance and support. Hardware support contracts, spares and replacement parts, set at 6% of capex a year.
• Staff. Operations engineers who rack, image, monitor and repair the fleet, set at $5,000 per node a year (for example, one fully loaded engineer at $250,000 per 50 nodes). The sweep from $2,500 to $10,000 moves the hourly cost from $0.036 to $0.143 per GPU-hour.

What renting costs: public prices checked on 25 October 2026

All prices below are list prices from public price pages or price APIs, read on 25 October 2026, converted to a per-accelerator-hour figure by dividing the instance price by the accelerator count the provider documents. They exclude taxes, storage, egress and support plans. Prices change; check the source before you rely on them.

NVIDIA Hopper generation

NVIDIA Blackwell generation

Microsoft's documentation states that each ND GB200 v6 VM has two Grace CPUs and four Blackwell GPUs, which is why that row divides by four.

AMD Instinct

Google TPU and AWS Trainium

Google's TPU pricing page also lists Dynamic Workload Scheduler prices: Ironwood is $6.00 per chip-hour in Flex-start mode and $8.40 in Calendar mode. We did not find a trn2 on-demand price in AWS's US East on-demand price data on the day we checked, so the Trainium2 row uses the Capacity Block rate.

How big the commitment discounts are

The discounts are consistent within each provider, and they are large:

A three-year reservation at 56% off is already pricing in much of what you would save by owning. That is the main reason the "buy versus rent" question is really three questions: buy versus on demand, buy versus a reservation, and reservation versus on demand.

Price per unit of peak compute

Comparing hourly prices across chips is meaningless without some normalization. The simplest one divides the price by the vendor's peak dense BF16 throughput. This is a peak figure, not measured performance: real throughput depends on model, precision, software and interconnect, and the only neutral measured comparisons are MLCommons MLPerf results and peer-reviewed papers.

Peak dense BF16 figures used here: H100 SXM 1,979 TFLOPS with sparsity on NVIDIA's product page, so about 989 dense; GB200 NVL72 lists 360 PFLOPS for 72 GPUs with the footnote that dense is one-half of the sparse figure shown, so 2.5 PFLOPS per GPU; AMD lists 1.3 PFLOPS dense BF16 for MI300X and 2.5 PFLOPS dense BF16 for MI355X; Google lists peak BF16 of 2,307 TFLOPS per Ironwood chip, 918 per Trillium chip and 459 per v5p chip.

Read this table as a starting point for a benchmark plan, not as a ranking. Peak BF16 says nothing about memory capacity (which sets how many chips a model needs), memory bandwidth (which bounds decode speed), interconnect (which sets how well training scales), FP8 and FP4 support, or how mature the software is for your model. Our GPU rental prices post covers why these prices move, and the cost per million tokens estimate shows how they translate into serving cost.

Worked example: the hourly cost of an owned H100 node

Here is the base case for one 8 x H100 node, using an all-in capex of $30,000 per GPU purely as a grid point.

Three things stand out.

First, the hourly cost when the node is fully busy, $1.72, is far below every on-demand price in the tables above. That gap is the cloud provider's margin plus the cost of the flexibility they sell you: instant capacity, no capex, no stranded hardware, and someone else's on-call rotation.

Second, electricity is a small line. Even the most expensive scenario we sweep (20 cents per kWh at a PUE of 1.54) raises energy to $0.275 per GPU-hour, adding $0.17 to the total, about 10% of F. Facility capacity per kW costs three times as much as the electricity in this example. Power matters enormously for whether you can get a site at all, but much less for the hourly cost once you have one.

Third, the capital line dominates. Getting the capex, the life and the cost of capital right matters more than anything else in the model.

The same node at different utilizations

Divide F by utilization to get the cost per useful GPU-hour:

At 20% utilization the owned GPU costs more per useful hour than AWS on demand ($6.88). At 30% it is already cheaper. The curve is a hyperbola: the first gains in utilization save the most money, and going from 80% to 90% saves only $0.24 per useful hour.

Break-even utilization, worked

Using u* = F / p across the capex grid and four on-demand H100 prices:

Even at the top of the grid, buying beats on-demand rental once average utilization passes about 37% against the cheapest on-demand H100 price we found. This is the headline most TCO articles stop at, and it hides the hard part: average utilization over four years, including the months of procurement, installation and burn-in before the first job runs, the weeks between projects, and the nights and weekends of an inference service.

For B200 nodes, with the 14.3 kW rating and the AWS on-demand price of $14.24 per GPU:

The break-even capex: the most you should pay

Inverting the formula gives the number to take into a negotiation: the maximum all-in capex per accelerator at which owning matches renting, for a target utilization and life.

max capex = (p x u - F0) x 8,760 / (CRF(r, n) + maintenance rate)

  F0 = the hourly fixed cost with zero capex (energy + facility + staff)
     = $0.480 per GPU-hour for the H100 node base case

Against AWS on-demand H100 at $6.88:

Against the AWS Capacity Block price of $5.97, four-year life: $46,176 at 40%, $75,076 at 60%, $103,975 at 80% and $125,650 at 95%.

Against the B200 on-demand price of $14.24, four-year life: $122,280 at 40%, $191,222 at 60%, $260,163 at 80% and $311,869 at 95%.

These numbers look generous, and that is the point. On-demand prices for the newest accelerators carry a large premium for flexibility and scarcity. If your all-in quote sits well under the break-even capex at a utilization you are confident of, buying is defensible on cost. If it only clears the bar at 90% utilization, you are betting the project on a utilization level few fleets sustain for four years.

Owning versus a reservation: utilization cancels

A reservation changes the shape of the problem because you pay for every hour of the term, used or not. Both options now have a fixed cost per hour, so the comparison is F against the reserved price q, and utilization affects both equally.

For the H100 node, matching the life to the term:

In words: with our facility, power, staff and financing assumptions, owning an H100 for three years is cheaper than Azure's three-year reservation whenever the all-in capex is below about $96,000 per GPU. That threshold is high because our facility and staff assumptions are lean. Raise the facility charge to $250 per kW per month, double the staff cost, and add a 15% contingency for failed parts and spares, and the threshold falls. The calculator below makes that easy to test.

What a reservation buys that owning does not:

• No procurement and installation lead time, so the first useful hour comes sooner.
• No residual value risk: at the end of the term you walk away.
• Operations, hardware failures and spares are the provider's problem.
• A path to the next generation without selling the old one.

What owning buys that a reservation does not: control over the network topology, the software stack and the data path, and the option to keep running fully depreciated hardware after the term for only its power, facility and staff costs, about $0.48 per GPU-hour in our base case.

Sensitivity: utilization, electricity and PUE

Energy cost per GPU-hour, H100 node at 70% of rated power

For an 8 x B200 node at PUE 1.2, the same four prices give $0.075, $0.145, $0.217 and $0.300 per GPU-hour.

Total fixed cost F at $30,000 capex per H100, four-year life

Across the whole electricity and PUE grid, F moves by about 14%, from $1.665 to $1.891. Across the utilization range from 30% to 90%, the cost per useful hour moves by a factor of three. That is the central lesson of accelerator TCO: utilization and capex are first-order, power price and PUE are second-order for cost (although power availability is first-order for whether you can build at all).

Power still matters at cluster scale. A 1,024-GPU H100 cluster (128 nodes) at 70% of rated power and a PUE of 1.2 draws about 1,097 kW at the facility meter, uses about 9,607 MWh a year and costs about $926,000 a year in electricity at 9.64 cents. Meta's Llama 3 paper also notes that tens of thousands of GPUs changing power together can swing data center power consumption "on the order of tens of megawatts", which is a grid engineering problem as much as a cost one.

Training fleet economics

Training is the workload that best fits owned hardware, because a well-run training cluster can stay busy. The compute needed for a dense transformer is about 6 x N x D floating point operations, where N is the parameter count and D the number of training tokens (Kaplan et al. 2020). Hoffmann et al. 2022 (Chinchilla) found compute-optimal training at roughly 20 tokens per parameter, which we use as the token budget here.

Achieved throughput is far below peak. Model FLOPs utilization (MFU) measures the useful fraction. Meta reported 38% to 43% BF16 MFU for Llama 3 405B, trained on up to 16K H100 GPUs. ByteDance's MegaScale paper reported 55.2% MFU training a 175B model on 12,288 GPUs. We also apply a 90% effective training time, close to the "higher than 90%" Meta reported, to account for restarts.

GPU-hours on H100 (989 TFLOPS dense BF16 peak), Chinchilla token budget:

What those GPU-hours cost:

The owned column assumes the cluster is busy every hour, and that is exactly what a single run cannot guarantee. A 100B Chinchilla run at 40% MFU needs about 935,751 GPU-hours, or 38 days on 1,024 GPUs. To keep that cluster busy for a year you need about nine and a half such runs, or a steady mix of ablations, fine-tuning and evaluation jobs to fill the gaps. A 10B run needs about 9,358 GPU-hours, six days on 64 GPUs. A team that trains one 10B model a quarter would leave an owned 64-GPU cluster idle about 93% of the time, and should rent.

The energy for that 100B run, at 70% of rated node power and PUE 1.2, is about 1,002 MWh, roughly $96,600 at 9.64 cents per kWh. That is under 2% of its on-demand compute bill.

Reliability is a hidden training cost on owned hardware. During a 54-day snapshot of Llama 3 pre-training, Meta recorded 466 job interruptions, 419 of them unexpected, and attributed about 78% of the unexpected ones to confirmed or suspected hardware issues. That is almost eight unexpected interruptions a day on a cluster of that size. On rented capacity, replacing the failed host is the provider's job; on owned capacity it is yours, and the spares, on-call staff and checkpointing engineering belong in F.

For the companion estimates by model size, see our posts on whether AI is getting cheaper per token and the self-hosting cost per million tokens.

Inference fleet economics

Inference demand follows people. Traffic rises during working hours, falls at night and dips at weekends, and a fleet sized for the peak sits partly idle the rest of the time. That is the case where a hybrid of owned and rented capacity usually beats either extreme.

Take an illustrative daily demand curve that swings smoothly between 0.5 and 1.5 times its average. A fleet sized for the peak runs at 1.0 / 1.5 = 66.7% average utilization. Now decide, one accelerator at a time, whether to own or rent. The k-th owned accelerator is busy only during the hours when demand exceeds k. Owning it pays off when the fraction of hours it is busy exceeds F / p, the same break-even ratio as before. So the optimal owned capacity is the level of demand that is exceeded for exactly F / p of the hours, and you rent everything above it.

demand
 1.5 |        ___
     |      /     \            rent the peak (above k)
  k  |-----/-------\--------------------------------
     |    /         \                 own the base (below k)
 1.0 |   /           \           /
     |  /             \         /
 0.5 |_/_______________\_______/____  hours of the day

Worked in Python for that demand curve:

Costs are in units of the on-demand price per unit of average demand. Read the middle row: when owned capacity costs half the on-demand price per hour, the best plan owns enough for average demand, runs that fleet at 84% utilization, and rents the 16% of demand above it. It costs 34% less than renting everything and 12% less than owning enough for the peak. When owning is relatively expensive (F / p of 0.7, for example against a deeply discounted reservation), owning for the peak costs more than renting everything, while the hybrid still saves 17%.

Three practical caveats make this harder in real life:

• Rented burst capacity has to exist when you need it. On-demand accelerator capacity is not guaranteed. Capacity blocks and scheduled modes such as Google's Dynamic Workload Scheduler exist precisely because it is not.
• Inference efficiency depends on batching. A busy accelerator serving large batches produces far more tokens per hour than one serving single requests, so utilization measured as "busy time" can overstate useful work. The cost per million tokens estimate shows how batch size changes cost per token.
• Latency targets cap utilization. Running a serving fleet at 95% busy usually means queueing delay. Most teams hold back headroom, which lowers achievable u for owned and rented fleets alike.

The generation treadmill

A four-year life is a bet that the accelerator stays economically useful for four years, and recent price lists show how quickly that bet ages. On Oracle's list, an H100 rents for $10.00 per GPU-hour and a GB200 GPU for $16.00. Per peak dense BF16 PFLOP-hour that is $10.11 against $6.40: the newer part is 37% cheaper per unit of peak compute, before counting FP4 support and the larger NVLink domain.

For an owner, that matters in two ways. The opportunity cost of an owned H100 is not the on-demand price of an H100 forever; it is the falling price of whatever compute could do the same work next year. And the residual value of the hardware at the end of its life, which we set to zero in this model, depends on what newer parts cost by then. A shorter life (three years) in the model is a simple way to price in obsolescence. Amazon's own move from six to five years for part of its server fleet points the same way.

The counterweight is that an owned, fully depreciated accelerator keeps running for only its energy, facility and staff cost, about $0.48 per GPU-hour in our H100 base case. Many inference and fine-tuning workloads do not need the newest part, and older accelerators often end their lives serving smaller models at low marginal cost.

Chips you can only rent

Google sells TPU capacity through Google Cloud, and AWS offers Trainium and Inferentia through EC2. We found no public purchase price for either, so the buy side of the comparison does not apply. What you can compare is on demand versus commitment:

• Google's TPU price page lists one-year commitments at 30% below on demand and three-year commitments at 55% below, for Ironwood, Trillium, v5p and v5e alike.
• Ironwood's on-demand price of $12.00 per chip-hour falls to $5.40 under a three-year commitment, or $2.34 per peak BF16 PFLOP-hour using Google's 2,307 TFLOPS figure.
• AWS lists trn2.48xlarge Capacity Blocks at $35.7608 per instance-hour for 16 Trainium2 chips, about $2.24 per chip-hour.

The commitment decision follows the same logic as ownership. A three-year commitment at 45% of the on-demand price beats on demand when your average use of the committed capacity exceeds 45%. That is the break-even utilization formula with F replaced by the committed price.

The calculator

This script holds every formula in this post. It uses only the Python standard library; we ran it on a Mac to produce the tables above. Swap the capex sweep for your quote, and the facility, staff and power inputs for your own.

"""AI accelerator TCO: buy versus rent, per accelerator-hour.

Every input is a stated assumption. Cloud prices were checked on 25 October 2026.
Capex values are a sweep grid, not quotes: plug in your own vendor quote.
"""

HOURS = 8760

def crf(rate, years):
    """Capital recovery factor: the annual payment that repays 1 unit of capex."""
    if rate == 0:
        return 1 / years
    g = (1 + rate) ** years
    return rate * g / (g - 1)

def owned_cost_per_hour(capex_per_gpu, node_rated_kw, gpus_per_node=8, years=4,
                        rate=0.08, maint=0.06, op_frac=0.70, pue=1.2,
                        usd_per_kwh=0.0964, colo_usd_per_kw_month=175,
                        staff_usd_per_node_year=5000):
    """Fixed cost per accelerator-hour (paid whether the chip is busy or idle)."""
    capital = capex_per_gpu * (crf(rate, years) + maint) / HOURS
    energy = node_rated_kw * op_frac * pue * usd_per_kwh / gpus_per_node
    facility = node_rated_kw * colo_usd_per_kw_month * 12 / (gpus_per_node * HOURS)
    staff = staff_usd_per_node_year / (gpus_per_node * HOURS)
    return {"capital": capital, "energy": energy, "facility": facility,
            "staff": staff, "total": capital + energy + facility + staff}

def breakeven_utilisation(fixed_per_hour, on_demand_price):
    return fixed_per_hour / on_demand_price

def breakeven_capex(on_demand_price, utilisation, node_rated_kw, years=4, **kw):
    zero = owned_cost_per_hour(0, node_rated_kw, years=years, **kw)["total"]
    rate, maint = kw.get("rate", 0.08), kw.get("maint", 0.06)
    return (on_demand_price * utilisation - zero) * HOURS / (crf(rate, years) + maint)

if __name__ == "__main__":
    h100_node_kw = 10.2    # DGX H100/H200 max system power, NVIDIA user guide
    b200_node_kw = 14.3    # DGX B200 max system power, NVIDIA user guide
    aws_p5_od = 55.04 / 8  # p5.48xlarge on-demand, us-east-1, 8 x H100
    aws_p6_od = 113.9328 / 8

    for capex in (20_000, 30_000, 40_000, 50_000):
        c = owned_cost_per_hour(capex, h100_node_kw)
        u = breakeven_utilisation(c["total"], aws_p5_od)
        print(f"H100 capex/GPU {capex:>6}: fixed ${c['total']:.2f}/h, "
              f"break-even utilisation vs AWS on-demand {u:.0%}")

    for u in (0.4, 0.6, 0.8):
        cap = breakeven_capex(aws_p5_od, u, h100_node_kw)
        print(f"Max all-in H100 capex/GPU at {u:.0%} utilisation (4 years): ${cap:,.0f}")

    for capex in (40_000, 60_000, 80_000):
        c = owned_cost_per_hour(capex, b200_node_kw)
        print(f"B200 capex/GPU {capex:>6}: fixed ${c['total']:.2f}/h, "
              f"break-even utilisation {breakeven_utilisation(c['total'], aws_p6_od):.0%}")

Its output on our machine:

H100 capex/GPU  20000: fixed $1.31/h, break-even utilisation vs AWS on-demand 19%
H100 capex/GPU  30000: fixed $1.72/h, break-even utilisation vs AWS on-demand 25%
H100 capex/GPU  40000: fixed $2.13/h, break-even utilisation vs AWS on-demand 31%
H100 capex/GPU  50000: fixed $2.55/h, break-even utilisation vs AWS on-demand 37%
Max all-in H100 capex/GPU at 40% utilisation (4 years): $54,986
Max all-in H100 capex/GPU at 60% utilisation (4 years): $88,291
Max all-in H100 capex/GPU at 80% utilisation (4 years): $121,596
B200 capex/GPU  40000: fixed $2.30/h, break-even utilisation 16%
B200 capex/GPU  60000: fixed $3.12/h, break-even utilisation 22%
B200 capex/GPU  80000: fixed $3.95/h, break-even utilisation 28%

A step by step TCO walkthrough for your own decision

1. Write down the workload, not the chip. Training runs per year and their GPU-hours, or inference demand by hour of day and the latency target. Utilization comes from the workload; everything else follows.
2. Collect three prices for the same accelerator. On demand, a one-year commitment and a three-year commitment, from at least two providers, with the date you checked. Divide by the accelerator count the provider documents.
3. Get an all-in quote. Server, scale-up fabric, network share, storage share, installation, spares and support, per accelerator.
4. Pick a life and a cost of capital. Run three and four years at least. If your finance team uses straight-line depreciation for reporting, still use a cost of capital for the decision.
5. Get the power numbers from the vendor's own manual. Rated system power for sizing; about 70% of rated for the energy bill, per LBNL, unless you have your own meter data.
6. Get the facility price per kW for the cooling type the hardware needs. Air-cooled 14 kW nodes and liquid-cooled racks are different facilities.
7. Compute F with the calculator, then the break-even utilization against on demand and the break-even capex against each commitment.
8. Stress the plan. Halve the first year's utilization for procurement and ramp-up, add a spares and failure allowance, and set residual value to zero. If owning still wins, the case holds up.
9. Plan the hybrid. Own the load you are confident of, commit for the next layer, and rent the peak.
10. Budget the software stack. Every owned node needs drivers, firmware, a CUDA or ROCm toolkit, communication libraries and container images kept current. That is staff time and a security process, covered below.

Common mistakes

• Comparing the owned cost at 100% utilization with the on-demand price. F is the cost per hour when the hardware is fully busy. The fair number is F / u, with u measured over the whole life, including ramp-up and idle weeks.
• Comparing owning with on demand when you would actually reserve. If your realistic alternative is a three-year reservation at 56% off, compare F with that price, not with on demand.
• Using rated power for the electricity bill. Rated power sizes the electrical system. LBNL models typical AI server draw at about 70% of rated.
• Ignoring PUE, or using a hyperscaler's PUE for a colocation cage. Google's 1.09 is a fleet figure from purpose-built sites; the Uptime survey average is 1.54.
• Leaving the network out of capex. The scale-out fabric for a training cluster is a large part of the bill and depreciates with it.
• Treating sparse throughput as dense. NVIDIA's H100 and GB200 pages list several Tensor Core figures with sparsity; dense is half. Normalizing price by a sparse figure flatters every chip equally and misleads when comparing with vendors that publish dense figures.
• Treating peak throughput as performance. Achieved MFU in published large-scale training runs is 38% to 55%. Use MLPerf results or your own benchmark for the chips you shortlist.
• Assuming a six-year life for accelerators. Even a large cloud provider shortened part of its server fleet to five years. Newer chips rent for far less per unit of compute.
• Forgetting failures. At scale, hardware interruptions are a daily event. Spares, staff and checkpointing are real costs of owning.
• Forgetting that the software stack is part of the asset. An owned cluster that runs an unpatched driver, an old CUDA toolkit or stale container images is a security and reliability liability that no hourly price reflects.

FAQ

Is it cheaper to buy or rent GPUs for AI?

It depends on utilization. In our worked example, an owned H100 with an illustrative all-in capex of $30,000 costs about $1.72 per GPU-hour when fully busy, so it beats the $6.88 AWS on-demand price above about 25% average utilization. Against a three-year reservation, which also charges for idle hours, the comparison depends on capex and operating costs rather than on utilization.

What is break-even utilization for an AI accelerator?

It is the average utilization at which owning costs the same per useful hour as renting on demand: the owned fixed cost per hour divided by the on-demand price. If owning costs $2 per hour and renting costs $8, the break-even is 25%.

How much does it cost to run an H100 per hour on your own hardware?

Excluding capex, our base case is about $0.48 per GPU-hour: $0.10 of electricity at 9.64 cents per kWh and PUE 1.2, $0.31 of facility capacity at $175 per kW per month, and $0.07 of staff. Capital recovery and maintenance add about $0.41 per hour for every $10,000 of all-in capex over four years at 8%.

How much power does an 8-GPU H100 or B200 server use?

NVIDIA's user guides list 10.2 kW maximum for a DGX H100 or H200 and 14.3 kW maximum for a DGX B200. Typical draw under AI workloads is lower; LBNL models it at about 70% of rated, based on measured H100 nodes averaging 74%.

Does electricity price matter much in GPU TCO?

Less than most people expect. In our H100 example electricity is about 6% of the hourly cost, and moving from 5 to 20 cents per kWh with a worse PUE changes the total by about 14% at most. Power availability matters much more than its price, because it decides whether you can build at all.

What PUE should I assume for an AI data center?

Use your facility's measured figure. For planning, Google reports 1.09 for its own fleet in 2025, the Uptime Institute survey average was 1.54, and LBNL estimates the US average at 1.4 in 2023 and projects 1.15 to 1.35 by 2028.

Is liquid cooling required for Blackwell GPUs?

Not for every system. NVIDIA's DGX B200 is air cooled, with 1,550 CFM of airflow and 48,794 BTU per hour of heat output listed in its user guide. The GB200 NVL72 and GB300 NVL72 racks are liquid cooled, which NVIDIA says increases density and reduces floor space.

Over how many years should AI accelerators be depreciated?

Three to five years is the common planning range for a buyer. Amazon moved a subset of its servers and networking equipment from six to five years from 1 January 2025. A shorter life is a reasonable way to price in the falling cost of newer compute.

How big are reserved instance discounts for GPUs?

On the prices we checked, Azure's ND H100 v5 reservations are 36% below pay as you go for one year, 56% for three years and 60% for five years. Google lists TPU commitments at 30% below on demand for one year and 55% for three years.

Can I buy Google TPUs or AWS Trainium chips?

We found no public purchase price for either. Both are offered as cloud capacity, so the relevant decision is on demand versus a commitment, which follows the same break-even logic as buying.

Where Safeguard fits

Safeguard does not design, program, price or speed up accelerators, and it does not lower GPU cost. Nothing in this TCO model depends on Safeguard. The fit is with the software every owned or rented accelerator runs: kernel drivers and firmware, CUDA or ROCm toolkits, NCCL or RCCL, framework builds, serving frameworks and the container images that bundle them. Those are versioned components with their own advisories, and our GPU software supply chain guide lists what to record for each.

Safeguard generates and analyzes SBOMs in SPDX and CycloneDX, scans repositories and container images (for example safeguard scan --image against a training or serving image), matches components against vulnerability databases, shows which projects carry a given library version, and ranks findings with reachability analysis. Autonomous remediation can open fix pull requests when a fixed version exists, and it ships switched off. What Safeguard does not cover here: host drivers and firmware outside any image you scan, the hardware itself, and the cloud provider's own infrastructure under a rented instance. Those updates come from your Linux distribution, the accelerator vendor and the provider.

Whether you buy or rent, an AI fleet is rebuilt constantly, and every rebuild pulls a fresh set of Python wheels, toolkit libraries and base images that may carry new advisories. The steps below apply to those dependencies and images, not to the silicon.

More in this series

Other posts in our AI chips and costs series:

• AI Chips Compared 2026: B200, H200, MI355X, Gaudi 3, TPU, Trainium
• Cheapest LLM Inference: How to Cut Cost per Million Tokens
• Fastest LLM Inference: Lowest Latency and Most Tokens per Second
• How Many GPUs Does a 1 Trillion Parameter Model Need?
• How to Estimate AI Training and Inference Cost: Formulas and a Calculator
• LLM Inference Cost by Chip and Model Size, 1B to 1 Trillion Parameters
• LLM Training Cost by Chip and Model Size: 1B to 1T, Worked

Fix it without waiting for upstream

Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.

1. Autonomous remediation: pull requests, not tickets

When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.

2. Gold Open Source: the safe version in seconds, free

gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.

3. Gold Registry: start from components that are already fixed

registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.

When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.

One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.

See how Safeguard would help

The first demo is a real Safeguard CLI session. Asked to fix a lodash CVE, it reads the finding and the advisory, asks before any command that changes something, edits package.json, runs npm install and npm test, and pushes nothing. The same flow applies to the requirements file or Dockerfile of a training or inference service, whether it runs on owned nodes or rented instances.

Click through it at your own pace, or open it full screen.

The second demo sets up the local runner on your own machine, approves it with a device code, and then scans from the terminal. That suits teams that build accelerator images on internal hosts next to an owned cluster and want to scan them where they are built.

Click through it at your own pace, or open it full screen.

The third demo connects Claude to the Safeguard MCP server with OAuth and chooses which tools Claude may use. While planning a fleet change, you can ask in the same conversation which images still carry a library version with an open advisory.

Click through it at your own pace, or open it full screen.

To try the CLI yourself, download Safeguard and follow the CLI documentation.

Quick check

Sources

• AWS, Amazon EC2 On-Demand pricing (price data for US East, checked 25 October 2026)
• AWS, Amazon EC2 Capacity Blocks for ML pricing (checked 25 October 2026)
• Microsoft, Azure Retail Prices API (queried 25 October 2026)
• Microsoft Learn, ND H100 v5 sizes series
• Microsoft Learn, ND GB200 v6 sizes series
• Microsoft Learn, ND MI300X v5 sizes series
• Oracle, Cloud price list API (queried 25 October 2026)
• Google Cloud, Cloud TPU pricing (checked 25 October 2026)
• Google Cloud, TPU7x (Ironwood) documentation
• NVIDIA, DGX H100/H200 User Guide: Introduction
• NVIDIA, DGX B200 User Guide: Introduction
• NVIDIA, H100 Tensor Core GPU product page
• NVIDIA, GB200 NVL72 product page
• AMD, Instinct MI300X accelerator specifications
• AMD, Instinct MI355X accelerator specifications
• US Energy Information Administration, Electric Power Monthly, Table 5.6.A
• Google, Data center efficiency (PUE)
• Shehabi et al., 2024 United States Data Center Energy Usage Report, Lawrence Berkeley National Laboratory
• Amazon.com, Inc., Form 10-K for fiscal year 2024 (SEC EDGAR)
• Llama Team, AI at Meta, The Llama 3 Herd of Models (arXiv 2407.21783)
• Jiang et al., MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs (arXiv 2402.15627, USENIX NSDI 2024)
• Kaplan et al., Scaling Laws for Neural Language Models (arXiv 2001.08361)
• Hoffmann et al., Training Compute-Optimal Large Language Models (arXiv 2203.15556)

Last reviewed: 25 October 2026. Cloud prices are public list prices checked on that date and change often; capex figures are sweep values, not quotes; all costs are estimates built from the stated assumptions.