Air vs liquid cooling for AI servers: direct-to-chip and immersion

Air cooling moves heat from servers with fans and chilled air. Liquid cooling moves it with a fluid, either piped to cold plates on the chips (direct-to-chip) or by submerging hardware in a non-conductive liquid (immersion). Dense AI racks are pushing operators toward liquid, because liquid carries heat away far more effectively than air.

This article explains the three approaches in plain terms, what the public sources say about where each fits, and where the evidence is thin. Figures are labeled by source and date. Nothing here is a forecast or investment advice.

Key numbers at a glance

• 72 GPUs and 36 CPUs: contents of one NVIDIA GB200 NVL72 rack, which the company describes as a "rack-scale, liquid-cooled design" (NVIDIA product page, accessed 19 October 2026).
• 25x: NVIDIA's claim for performance at the same power versus an H100 air-cooled setup, from a test scenario the company marks as "projected performance subject to change" (NVIDIA product page, accessed 19 October 2026).
• 1.6 falling to 1.4: average power usage effectiveness (PUE) of US data centers from 2014 to 2023 (Lawrence Berkeley National Laboratory, report dated December 2024).
• 1.15 to 1.35: LBNL's scenario range for average US PUE in 2028, driven partly by more liquid-cooled AI servers (same report).
• 30 percent: share of US data center electricity used by infrastructure such as cooling and power conversion in 2023, down from 40 percent in 2014 (same report).
• 325 to 580 TWh: LBNL's range for US data center electricity use in 2028, with cooling choices one of the scenario variables (same report). The 2023 figure was 176 TWh.
• 1.09: Google's reported fleet-wide annual PUE for 2025, compared by Google with a 1.54 industry survey average from the Uptime Institute (Google efficiency page, accessed 19 October 2026, company statement).
• 1 MW: single-rack power that the Open Compute Project says AI data centers are "on track to reach and then surpass" (OCP Foundation post, 12 October 2026).
• Up to 3x: heat removal advantage Microsoft reported for microfluidic cooling over cold plates in lab-scale tests (Microsoft feature article, no publication date shown on the page, accessed 19 October 2026).

Why heat is the problem

Every watt that goes into a server comes out as heat. The Open Compute Project's heat reuse group puts it plainly: almost 100 percent of the energy used in a processor turns into heat. A data center is, in the end, a building that turns electricity into computation and then has to get rid of the leftover heat before the chips overheat and slow down or fail.

For most of the history of computing, blowing cool air across the hardware was enough. Racks held general-purpose servers that each drew a few hundred watts. A rack, the standard cabinet that holds a stack of servers, drew perhaps a few kilowatts to a few tens of kilowatts. A kilowatt (kW) is a thousand watts, roughly the draw of a small space heater.

AI changes the arithmetic. Training and running large models relies on graphics processing units (GPUs) and similar accelerators. These chips draw much more power than ordinary processors, and operators pack many of them close together so that they can talk to each other quickly. That makes each rack far hotter than a conventional one.

The consequence is simple physics. Air holds little heat per unit of volume. To carry away a large amount of heat with air alone, you need to move a great deal of it, which means large fans, wide aisles and a lot of fan energy. At some point it stops being practical. Liquids hold much more heat per unit of volume, so a small pipe of water can move what would take a hurricane of air.

How air cooling works

Air cooling is the long-standing default, and it remains common. In a typical room, chilled air is delivered to the front of the racks, servers pull it through, and the heated air leaves the back. The hot air is returned to cooling equipment, cooled again and recirculated.

Operators use several tricks to make this efficient. Hot aisle and cold aisle layouts keep cold and hot air from mixing. Containment, meaning physical barriers or curtains around an aisle, keeps them apart even better. The point of both is to stop hot exhaust air from finding its way back into server intakes, which would force the cooling plant to work harder for the same result.

The heat then has to leave the building. LBNL's 2024 report lists the main systems it modeled for US facilities:

• Direct expansion units, often called computer room air conditioners, which pass refrigerant through indoor coils. These are common in small and midsize rooms because they are simple and low cost.
• Air-cooled chillers, which make cold water or refrigerant and reject heat to outdoor air with condensers and fans. They use no water at the site but use more electricity.
• Water-cooled chillers with cooling towers, which are usually more energy efficient but consume water through evaporation.
• Airside economizers, which bring in outside air when the weather is cool enough, sometimes helped by evaporative or adiabatic assist, meaning a spray of water that cools the air as it evaporates.

The key trade-off here, which the next sections build on, is that the more water-based evaporation you use, the less electricity you tend to use. Cooling choices are a bargain between energy and water. LBNL says this directly: water-cooled chillers and other evaporation-based systems are generally more energy efficient than air-cooled chillers or other waterless systems, while air-cooled chillers use no water but use more energy.

Air cooling has a ceiling. Racks designed around air can be pushed higher with better containment and rear-door heat exchangers, which are radiators mounted on the back of a rack that cool exhaust air with liquid. But modern AI racks go beyond what air can reasonably handle, which is why NVIDIA describes its flagship rack as a liquid-cooled design.

How direct-to-chip cooling works

Direct-to-chip cooling, also called direct liquid cooling or cold plate cooling, brings liquid right up to the hottest components. A metal plate with tiny internal channels, the cold plate, sits on top of each processor or GPU. Coolant circulates through the plate, picks up heat from the chip, and carries it away through small tubes inside the server and out to the rack.

The Open Compute Project (OCP), an industry group that publishes shared hardware designs, describes cold plate cooling as one of five areas of its Cooling Environments project and lists standard interfaces for "the technology cooling system, from cold plates to CDUs," including tubing, manifolds, quick disconnects and coolants (OCP project page, accessed 19 October 2026).

Two terms are worth explaining.

A coolant distribution unit (CDU) is the heat-exchange hub between the liquid loop that touches the servers and the larger facility water system. It pumps coolant to racks, keeps it clean and at the right temperature, and passes the heat on to the building. The two loops stay separate, so facility water never touches server hardware.

A manifold is the plumbing at the rack that splits the coolant supply among many servers and collects it again. Quick disconnects are the dripless couplings that let a technician unplug a server without spilling.

NVIDIA's developer blog describes the GB200 compute tray as including "cold plates and connections for liquid cooling." That is direct-to-chip in practice. It is worth noting that direct-to-chip does not remove all heat by liquid. Memory, power supplies and networking parts often still rely on airflow, so many designs are hybrid, with liquid for the hottest chips and fans for the rest. LBNL's description of its AI assumptions is consistent with this, calling the cooling systems used for AI servers "a mix of liquid and air cooling."

There are two flavors of direct-to-chip. In single-phase systems, the liquid stays liquid and simply warms up. In two-phase systems, a low-boiling-point fluid boils on the cold plate and the vapor carries away heat as it condenses elsewhere. The OCP's October 2026 post describes the transition to two-phase direct-to-chip cooling as "looming," with no date or power figure given, and notes that it delivered a version 1.0 specification for its Deschutes CDU along with guidance on coolant liquids.

A frontier beyond cold plates is cooling inside the chip package itself. Microsoft described a microfluidic approach in which tiny channels are etched into the back of the silicon so coolant flows directly onto the chip. In lab-scale tests, it reported heat removal up to three times better than cold plates, and a 65 percent reduction in the maximum temperature rise of silicon inside a GPU, with both figures varying by workload and chip type. Microsoft's own article is clear that these are lab results and that reliability testing and production manufacturing remain open work. Treat it as research, not a deployed product.

How immersion cooling works

Immersion cooling puts the hardware itself into a liquid. The liquid must be dielectric, meaning it does not conduct electricity, so it is safe to touch live electronics. The OCP's cooling project lists a base specification for two-phase dielectric heat transfer fluids among its documents, which shows that fluid chemistry is itself an area being standardised.

There are again two main kinds:

• Single-phase immersion. The fluid never boils. It absorbs heat from the servers, is pumped to a heat exchanger and returns. Because it does not boil, the temperature range is wide and outdoor conditions can be used for cooling more effectively.
• Two-phase immersion. A low-boiling-point fluid, typically a fluorocarbon, boils at the hot components. The vapor rises to a condenser plate, gives up heat and drips back down.

Physically, a bath can be an open tank holding many servers upright, or sealed chassis that circulate fluid inside a conventional rack. In a fully immersed design, server fans and much of the room air handling are no longer needed, and pumps and heat exchangers or dry coolers take their place.

Immersion has real hurdles. Fans must be removed from servers. Some thermal pastes can degrade in certain fluids. Heavy components may need lifting equipment to pull from a tank. Maintenance practices change, since technicians work with fluid rather than open air. And not every server design is built for it, so buyers need hardware certified for the fluid chosen.

There is no reliable public count of immersion deployments. A fair reading of vendor and industry material is that immersion is far less common than air cooling and, in AI, currently less discussed than direct-to-chip. The OCP project page lists immersion as one of its five cooling work areas, with a scope covering hardware and component support, reference designs and interoperability, and sustainability.

Rack density: why it drives the choice

Rack density, the power drawn by one rack, is the number that most often decides the cooling method. A few reference points from published sources:

• LBNL's 2024 report notes that 72-GPU rack-scale designs such as the GB200 were announced but "not yet being deployed" when it was written, and that their power draw was expected to scale close to linearly from 8-GPU machines. The report is dated December 2024, so deployments have moved since.
• NVIDIA's GB200 NVL72 product and developer pages describe the rack as liquid-cooled but do not state a rack power or cooling capacity. Figures quoted elsewhere for this rack are not confirmed by the manufacturer, so none is used here.
• The OCP said in October 2026 that power densities are on track to reach and then surpass 1 MW in a single rack. The same post says that in settings such as neoclouds, colocation, telecom points of presence and industrial sites, a 100 kW rack is unlikely to deploy soon. Both statements are the OCP community's view, not a measured market average.

The practical reading is that there is a wide spread. A rack of ordinary servers still sits in the tens of kilowatts and is happily air-cooled. The newest AI racks sit in the range where liquid is expected. A facility designed for AI at scale may therefore run both, with liquid for GPU halls and air for storage and networking.

A related point: a data center building is designed around a power and cooling budget per hall. Putting a 100 kW rack into a room designed for 10 kW racks is not a matter of swapping hardware. It involves the power feed, the floor load, the piping and the heat rejection plant. That is one reason the shift is gradual and why retrofits are harder than new builds.

What each approach does to energy and water

Cooling affects two resource bills, electricity and water, and the effect differs by approach.

On energy, LBNL reports that infrastructure energy, which includes cooling and power conversion, fell from 40 percent of US data center electricity in 2014 to 30 percent in 2023 as average PUE improved. PUE is total facility electricity divided by the electricity used by the IT equipment alone. A PUE of 1.4 means that for every kilowatt-hour of computing, 0.4 kWh goes to cooling and other overhead. LBNL attributes the drop mainly to the shift toward larger hyperscale and colocation facilities and expects average PUE to fall to between 1.15 and 1.35 by 2028, with more liquid-cooled AI servers as one driver.

Why would liquid help? LBNL explains that liquid-cooled IT equipment can often run at higher water or refrigerant temperatures than air-cooled equipment, which means "less chiller run hours and less adiabatic assist." Its simulations assume an ASHRAE liquid cooling class called W45, a facility water supply temperature class that tolerates warm water up to 45 degrees Celsius. Warm return water can be cooled by outdoor air through a dry cooler, a radiator-like device, for much of the year in many climates, without running compressors.

On water, the picture is more mixed than many headlines suggest. Liquid cooling inside the rack uses a closed loop that does not itself consume water. What matters is how the heat then leaves the building. If it goes to a dry cooler, site water use is near zero. If it goes to an evaporative cooling tower, water is consumed. LBNL notes that its waterside economizer case for liquid-cooled IT "consumes substantial amounts of water, like all evaporative cooling systems." So liquid to the rack does not by itself settle the water question. The heat rejection choice does.

Microsoft's design shows the closed-loop end of the range. It says its datacenter design in use since August 2024 "consumes zero water for cooling," using chip-level cooling in a closed loop that is filled during construction and circulates between servers and chillers without a fresh supply. The company also states the cost: replacing evaporative cooling with mechanical cooling raises PUE, and it expects a "nominal increase" in annual energy use across its fleet. Pilot sites are planned for 2026 in Phoenix, Arizona and Mt. Pleasant, Wisconsin, with new sites expected to begin coming online in late 2027, according to the company. These are company statements, not independent measurements.

For the water side in more detail, see the companion piece on data center water use, direct and indirect.

Choosing between them: a plain decision guide

No source supports a single best answer, and the right choice depends on the site. These are the questions operators ask:

• How dense are the racks? Below roughly the range that air handles comfortably, air is simpler and cheaper to operate. Above it, liquid becomes close to a requirement.
• Is this a new build or a retrofit? New halls can be designed around piping and CDUs. Existing air-cooled rooms can sometimes add rear-door heat exchangers as a stepping stone.
• What does the climate allow? Warm-water liquid loops can reject heat to outdoor air more of the year in cool climates, which lowers both energy and water. The post on why location changes cooling, water and power covers this.
• What does the local water situation allow? Where water is scarce or politically sensitive, dry heat rejection may be chosen even at an energy cost.
• What does the hardware vendor support? Warranty and design rules often decide between direct-to-chip and immersion, since not all servers are built for both.
• Who will operate it? Liquid brings new failure modes such as leaks and pump faults, and new skills for staff.

For the supply side of this equipment, including who makes chillers and CDUs and what filings say about orders, see AI data center cooling: chillers, CDUs and liquid cooling supply. For the money, see AI data center cost per megawatt.

What is firm and what is uncertain

Firm, from primary or official sources:

• NVIDIA describes its GB200 NVL72 as liquid-cooled and its compute tray as using cold plates.
• LBNL reports a fall in average US PUE from 1.6 to 1.4 between 2014 and 2023 and models a further fall by 2028.
• LBNL and Microsoft both state that liquid cooling can raise coolant temperatures and reduce reliance on chillers, and that waterless cooling usually costs electricity.
• OCP is publishing open specifications across cold plates, CDUs, immersion, rear-door heat exchangers and heat reuse.

Uncertain or contested:

• Rack power figures. NVIDIA's pages do not state them for the NVL72, and secondary figures are not confirmed by the manufacturer.
• The 25x claim. NVIDIA's comparison is against an older air-cooled system in a test scenario, and it mixes the effect of the chip generation with the effect of cooling. It should not be read as the benefit of liquid cooling alone.
• Adoption. No primary source cited here gives a share of racks that are liquid-cooled today.
• Immersion's future role. Public evidence is thin and the terms of competition with direct-to-chip are unsettled.
• Microfluidics. Lab results only.
• PUE comparisons. PUE definitions differ between air and immersion designs, so the metric does not compare the two approaches cleanly. PUE also does not account for local climate, so a site in a cool place can post a lower figure without a better design.

What to watch next

• Whether the OCP's two-phase direct-to-chip and 1 MW rack work turns into shipped products and published specifications.
• Whether operators report liquid-cooled share of capacity in their sustainability reports.
• Whether Microsoft's 2026 pilot sites in Phoenix and Wisconsin report measured water and energy results after they come online.
• Retrofit announcements that add liquid to existing air-cooled halls.
• Published water and energy figures that separate the loop to the rack from the heat rejection plant, which would make comparisons much cleaner.

For the software running on this hardware, Safeguard offers SBOM and dependency scanning, with a free scan.

Quick check

Sources

• NVIDIA GB200 NVL72 product page, accessed 19 October 2026
• NVIDIA developer blog on GB200 NVL72, accessed 19 October 2026
• Lawrence Berkeley National Laboratory, 2024 United States Data Center Energy Usage Report (eScholarship), December 2024, read 19 October 2026
• Open Compute Project, Cooling Environments project, accessed 19 October 2026
• Open Compute Project, Realizing the OCP Open Data Center Ecosystem Vision, 12 October 2026
• Microsoft, next-generation datacenters that consume zero water for cooling, accessed 19 October 2026
• Microsoft, microfluidic cooling for AI chips, accessed 19 October 2026
• Google data center efficiency page, accessed 19 October 2026

Last reviewed: 19 October 2026.