Monday, April 18, 2022

"Nvidia’s Next GPU Shows That Transformers Are Transforming AI" (NVDA)

 I don't think NVDA has seen its lows:

https://api.wsj.net/api/kaavio/charts/big.chart?nosettings=1&symb=nvda&uf=0&type=2&size=2&sid=129254&style=320&freq=1&entitlementtoken=0c33378313484ba9b46b8e24ded87dd6&time=8&rand=1952884984&compidx=&ma=0&maval=9&lf=1&lf2=0&lf3=0&height=335&width=579&mocktick=1

 BigCharts

And I'm not sure their plan to be the engine that powers the metaverse will ever amount to anything.

And I know it is not as interesting as it was in 2015 - 2016 when we started touting it at $26 to $36 ($6.50 to $9 accounting for last year's 4:1 split)*

But it still has some of the most amazing tech out there.

From IEEE Spectrum, April 8:

The neural network behind big language processors is creeping into other corners of AI

Transformers, the type of neural network behind OpenAI’s GPT-3 and other big natural-language processors, are quickly becoming some of the most important in industry, and they are likely to spread to other—perhaps all—areas of AI. Nvidia’s new Hopper H100 is proof that the leading maker of chips for accelerating AI is a believer. Among the many architectural changes that distinguish the H100 from its predecessor, the A100, is a “transformer engine.” Not a distinct part of the new hardware exactly, it’s a way of dynamically changing the precision of the calculations in the cores to speed up the training of transformer neural networks.

“One of the big trends in AI is the emergence of transformers,” says Dave Salvator, senior product manager for AI inference and cloud at Nvidia. Transformers quickly took over language AI, because their networks pay “attention” to multiple sentences, enabling them to grasp context and antecedents. (The T in the benchmark language model BERT stands for “transformer” as it does in the occasionally insulting GPT-3.)

“We are trending very quickly toward trillion parameter models” —Dave Salvator, Nvidia

But more recently, researchers have been seeing an advantage to applying that same sense of attention to vision and other models dominated by convolutional neural networks. Salvator notes that more than two-thirds of papers about neural networks in the last two years dealt with transformers or their derivatives. “The number of challenges transformers can take on continues to grow,” he says....

....MUCH MORE
*As you can see on the stock price chart the all-time high was over $345 (346.47), equivalent to $1385 per share on the old stock. Here's one of many, I mean many, posts:
April 13, 2016
Our standard NVDA boilerplate: We don't do much with individual stocks on the blog but this one is special.

$36.28 last, passing the stock's old all time high from 2007, $36.00.

In 2017 Oak Ridge National Laboratory is scheduled to complete their newest supercomputer powered by NVIDIA Graphics Processing Unit chips and retake the title of World's Fastest Computer for the United States.

In the meantime NVDA is powering AI deep learning and autonomous vehicles and virtual reality and some other stuff.

$216.58 up $4.00 last