Wednesday, March 6, 2024

(very) Large Language Models: "SambaNova Trains Trillion-Parameter Model to Take On GPT-4"

From Electrical Engineering Times, March 6:

SambaNova has built and pre-trained a trillion-parameter AI model it calls Samba-1, designed for enterprises to customize and fine-tune with their own data. There are only a few trillion-parameter models in the world today, SambaNova CEO Rodrigo Liang told EE Times, but SambaNova’s intent is to enable enterprises to have their own trillion-parameter large language model (LLM) trained on their own data—without having to compromise data security.

“Samba-1 is an enterprise-class, trillion-parameter model that customers can train with their private data, without ever having to disclose [their data] into the public domain,” he said.

“Our goal is for every enterprise to have their own custom version of a trillion-parameter GPT,” he added. “If you think about how people will start customizing this, within a short time every company will have a different trillion-parameter model. But because they selected a different combination of experts, and they customized some, fine-tuned others…now instead of having only two trillion-parameter models in the world, you’ve got 100.”....

....MUCH MORE

Shades of Nvidia with their Every, town, every village, every hamlet, every wide spot in the road, should have their own (NVDA-powered) supercomputer.