"The computing power needed to train AI is now rising seven times faster than ever before" (NVDA)

It's getting pretty energy intensive as well.
From MIT's Technology Review, November 11:

In 2018, OpenAI found that the amount of computational power used to train the largest AI models had doubled every 3.4 months since 2012.

The San Francisco-based for-profit AI research lab has now added new data to its analysis. This shows how the post-2012 doubling compares with the historic doubling time since the beginning of the field. From 1959 to 2012, the amount of power required doubled every two years, following Moore’s Law. This means the doubling time today is more than seven times the previous rate.

This dramatic increase in the resources needed underscores just how costly the field’s achievements have become. Keep in mind that the above graph shows a logarithmic scale. On a linear scale (below), you can more clearly see how compute usage has increased by 300,000-fold in the last seven years.

The chart also notably does not include some of the most recent breakthroughs, including Google’s large-scale language model BERT, OpenAI’s language model GPT-2, or DeepMind’s StarCraft II-playing model AlphaStar.

In the past year, more and more researchers have sounded the alarm on the exploding costs of deep learning. In June, an analysis from researchers at the University of Massachusetts, Amherst, showed how these increasing computational costs directly translate into carbon emissions.....MORE

Related at ZD Net, Nov. 12:
Facebook’s latest giant language AI hits computing wall at 500 Nvidia GPUs