This was published the day before Moonshot AI announced their open-source Kimi K3. We link to it because we are trying to keep tabs on Ms. Murati.
I'm pretty sure the writer wasn't aware Kimi K3 was just around the corner but I wonder: Was the CIA aware of Moonshot's impending release? And if not, why not?
From The Deep View, July 16:
China and Europe have typically led the charge on open models. Now, another US company is joining the fray with a significant entrant into the race.
On Wednesday, Thinking Labs, the AI startup founded by ex-OpenAI CTO Mira Murati, launched its first model: Inkling. The model sets itself apart from leading AI labs in the US in a very important way: it is open-weight, allowing customers to customize it directly to their needs through fine-tuning and training.
To suit that purpose, the model was built to perform across a wide range of areas. Or, as Thinking Machines describes it, it's a "generalist model" that can reason about text, images, and audio rather than being proficient in just one domain. This is evident in benchmark performance, which isn't best-in-class but is consistent across specialties, as shown in the image below.
Other model features include:
- Parameters: Mixture-of-Experts transformer model with 975 billion total parameters, 41 billion active.
- Context window: Up to one million tokens
- Pretrained: On 45 trillion tokens of text, images, audio and video, allowing it to be proficient across all three.
- Lightweight counterpart: Inkling-Small, a lighter-weight model with 12 billion active parameters.
- Fine-tuning: To make customization accessible, the company says it is making Inkling available on Tinker, the company's training API platform.
- Cost-efficiency: Inkling's "controllable thinking effort" allows users to customize the cost/performance curve.
Until now, most open-source models have been developed in other countries, with China leading the way. These models are good for the ecosystem because they allow people and organizations to more deeply specialize the models, adjust their behavior, run them locally, and even save money. Though, as Thomas Randall, Research Director at Info-Tech Research Group, told The Deep View, they may not be for everyone....
....MUCH MORE
Previously on Thinking Labs:
May 28 - AI: "The $2B Mira Murati Mystery".
June 10 - "Mira Murati Unveils Her Startup’s A.I. Model in First Interview Since OpenAI"
....Unlike many A.I. founders who climbed traditional computer science or venture capital ranks, Murati’s path is rooted in product management. She studied mechanical engineering at Dartmouth College, where she worked on building a hybrid race car. After graduating, she joined Tesla in 2013 as a product manager for the Model X. She later led product and engineering at augmented reality startup Leap Motion (now UltraLeap), focusing on motion-based human-computer interaction.
In 2018, Murati joined OpenAI as head of applied A.I. and partnerships, rising quickly to become chief technology officer. Over six years, she helped steer the deployment of some of the most influential A.I. products, including ChatGPT, DALL-E, and GPT-4....
Other than all that, what's she ever done?
If interested see also:
November 2023 - More On Q* and Q-Learning
October 2024 - "OpenAI Executives Say AI Will Be Able to Do Any Job Within 10 Years"
August 2025 - "Thanks for Your $1 Billion Job Offer, Mark Zuckerberg. I’m Gonna Pass." (META)
She is one of the reasons we note re: SoftBank's all-in wager on OpenAI:
...Should SoftBank be unable to repay or refinance the debts it is taking on, the risk goes from theoretical to kaboom pretty fast and all the other daisy-chain financings get stress-tested in a real-world cascade.
And unfortunately chatbots in general and OpenAI/Sam Altman in particular may not be the future that Mr. Son seems to think....
And:
...Throw in the fact that OpenAI and their ChatGPT may not be the ultimate winner of this unprecedented build-out and there are reasons to be hyper-aware. Stay tuned.
