Ahead of Today's NVIDIA Earnings: A Look at One of the Competitors (GOOG; NVDA)

We'll have much more on Google next week but for today the next-gen Tensor Processing Unit.

From Extreme Tech:

Google Announces 8x Faster TPU 3.0 For AI, Machine Learning

For the past few years, Google has been building its own TPU (Tensor Processing Units) to handle various processing tasks related to artificial intelligence and machine learning. Google first announced the existence of TPUs in 2016, but said it had been using them internally for more than a year. The company’s second-generation TPUs have made the news recently for significantly improved performance, and the third-generation hardware will apparently continue that trend.

According to Google CEO Sundar Pichai, the new TPU 3.0 pods are 8x more powerful than the Google TPU 2.0 pods that we’ve previously covered, and Pichai declared that they’re power-hungry enough to need water cooling — something previous TPUs simply haven’t required. TechCrunch’s image shows a copper-plated cooler system, with water from the same pipe run through all four cooling plates.

So what do we know about TPU 3.0? Not much — but we can make a few educated guesses. According to Google’s own documentation, TPU 1.0 was built on a 28nm process node at TSMC, clocked at 700MHz, and consumed 40W of power. Each TPU PCB connected via PCIe 3.0 x16.

TPU 2.0 made some significant changes. Unlike TPU v1, which could only handle 8-bit integer operations, Google added support for single-precision floats in TPU v2 and added 8GB of HBM memory to each TPU to improve performance. A TPU cluster consists of 180 TFLOPS of total computational power, 64GB of HBM memory, and 2,400GB/s of memory bandwidth in total (the last thrown in purely of the purposes of making PC enthusiasts moan with envy).

Unlike TPU v1, which used 3.5-inch drive bays as its form factor, TPU v2 was welded together in groups of four ASIC chips. Google currently deploys TPUs in clusters of up to 64 boards, at 11.5 PFLOPS per cluster and 4TB of total HBM storage....MORE

As noted in last month's ""UPDATED—NVIDIA Wants to Be the Brains Behind the Surveillance State (NVDA)" there is so much power running through theTSU's and  GPU's that even the most energy-efficient require water-cooling.
Brave new world.