From IEEE Spectrum, October 13:
Here are four new ways to get it done
Walk into a typical data center and one of the first things that jumps out at you is the noise—the low, buzzing sound of thousands of fans: fans next to individual computer chips, fans on the back panels of server racks, fans on the network switches. All of those fans are pushing hot air away from the temperature-sensitive computer chips and toward air-conditioning units.
But those fans, whirr as they might, are no longer cutting it. Over the past decade, the power density of the most advanced computer chips has exploded. In 2017, Nvidia came out with the V100 GPU, which draws 300 watts of power. Most of that power dissipates back out as heat. Three years later, in 2020, Nvidia’s A100 came out, drawing up to 400 W. The now-popular H100 arrived in 2022 and consumes up to 700 W. The newest Blackwell GPUs, revealed in 2024, consume up to 1,200 W.
“Road maps are looking at over 2,000 watts [per chip] over the next year or two,” says Drew Matter, president and CEO of the liquid-cooling company Mikros Technologies. “In fact, the industry is preparing for 5-kilowatt chips and above in the foreseeable future.”
This power explosion is driven by the obvious culprit—AI. And all the extra computations consuming all that added power from advanced chips are generating unmanageable amounts of heat.
“The average power density in a rack was around 8 kW,” says Josh Claman, CEO of the startup Accelsius. “For AI, that’s growing to 100 kW per rack. That’s an order of magnitude. It’s really AI adoption that’s creating this real urgency” to figure out a better way to cool data centers.
Specifically, the urgency is to move away from fans and toward some sort of liquid cooling. For example, water has roughly four times the specific heat of air and is about 800 times as dense, meaning it can absorb around 3,200 times as much heat as a comparable volume of air can. What’s more, the thermal conductivity of water is 23.5 times as high as that of air, meaning that heat transfers to water much more readily.
“You can stick your hand into a hot oven and you won’t get burned. You stick your hand into a pot of boiling water and you can instantly get third-degree burns,” says Seamus Egan, general manager of immersion cooling at Airedale by Modine. “That’s because the liquid transfers heat much, much, much, much more quickly.”
The data-center industry by and large agrees that cooling chips with liquid is the future, at least for AI-focused data centers. “As AI has made racks denser and hotter, liquid cooling has become the de facto solution,” Karin Overstreet, president of Nortek Data Center Cooling, said via email.
But there are a number of ways to do liquid cooling, from the simple and straightforward to the complex and slightly weird.
At the simple end, there’s circulating chilled water through cold plates attached to the hottest chips. Then there’s circulating not water but a special dielectric fluid that boils inside the cold plate to take away the heat. A third approach is dunking the entire server into a fluid that keeps it cool. And, last and most splashy, is dunking the server into a boiling vat of liquid.
Which method will end up being the industry standard for the high-end AI factories of the future? At this point, it’s anyone’s guess. Here’s how the four methods work, and where they might find the most use
#1: Single-Phase Direct-to-Chip Cooling....
Also at IEEE Spectrum, October 16:
You Can Cool Chips With Lasers?!?!
Startup plans to cool data centers by converting heat to light
