Our last 'puter link of the day. From TechRadar, November 20:
Scientists are training a gargantuan one-trillion-parameter generative AI system dubbed 'ScienceGPT' based on scientific data from the newly established Aurora supercomputer.
The AuroraGPT AI model, which is being trained by researchers at the Argonne National Lab (ALN) in Illinois, USA, is powered by Intel's Ponte Vecchio GPUs which provide the main computing power, and is being backed by the US government.
Training could take months to complete, according to HPC Wire, with training currently limited to 256 of the roughly 10,000 nodes of the Aurora supercomputer, before this is scaled up over time. Even given this limitation, Intel and ANL are only testing the model training on a string of 64 nodes, with caution due to Aurora's unique design as a supercomputer.
Inside 'ScienceGPT'
At one trillion parameters, ScienceGPT will be one of the largest LLMs out there. While it won't quite hit the size of the reported 1.7-trillion-parameter GPT-4, developed by OpenAI, it'll be almost twice as large as the 560-billion-parameter Pathways Language Model, which powers Google's Bard....
....MUCH MORE
Previously on the Aurora supercomputer:
March 25: "The Computer That Will Change Everything"
November 18: The Computer That Will Change Everything: "New Breed of Supercomputer Aims for the Two Quintillion Mark"
