Cooling Is Eating the Rack

Every generation of AI hardware has a component that sneaks up on the industry. For the current one, it is not a chip at all. It is the plumbing.

An NVL72-class rack dissipates roughly 130 kW in the footprint of a refrigerator. Air cannot remove that heat — the physics gave out somewhere around 40 kW. So the flagship AI rack is now fully liquid-cooled as shipped: cold plates on every GPU and CPU, manifolds running the height of the rack, coolant distribution units (CDUs) marshaling flow, and facility water loops sized like district heating systems.

From feature to bill of materials

Liquid cooling used to be a retrofit sold to enthusiasts and HPC labs. It is now embedded in the accelerator bill of materials, and the numbers have gotten large enough to reprice an industry:

• Thermal specialists that were sleepy industrial suppliers five years ago now trade like AI infrastructure plays — Vertiv being the emblem.
• CDU lead times, cold-plate manufacturing capacity, and even quick-disconnect fittings have all had their shortage moments, each a miniature rerun of the GPU allocation drama.
• Data-center shells are being redesigned around water: pipe galleries, higher floor loading, and heat-rejection systems that determine site selection as surely as fiber routes once did.

The efficiency dividend nobody collects yet

Liquid cooling's dirty secret is that it is an opportunity disguised as a burden. Water leaving a dense AI hall at 40–50°C is a usable industrial input — district heating in Nordic deployments already monetizes it — but the overwhelming majority of AI heat is still rejected to the sky. The gigawatt campuses being sited today will exhaust city-scale quantities of low-grade heat, and almost none of them have an offtaker.

Meanwhile the next escalation is already on roadmaps: rack densities heading toward 250 kW and beyond, immersion and two-phase systems moving from pilot to spec sheet, and facility water consumption becoming a political flashpoint in drought-prone markets — which is pushing closed-loop and air-assisted designs even where water is technically cheaper.

The compute industry spent fifty years abstracting physics away from software. The rack is where physics is collecting its debt.