Showing posts sorted by date for query Google TPU. Sort by relevance Show all posts
Showing posts sorted by date for query Google TPU. Sort by relevance Show all posts

Tuesday, September 8, 2026

Google's TPU Chips Will Deliver Up To 50% Better Perfomance Per Dollar On Some Inference Chores Vs. Nvidia (GOOG; NVDA)

Took ya long enough.*

From SemiAnalysis, September 7:

  • TPU Inference Externalization Full Steam Ahead - InferenceX
  • InferenceX, Up to 50% Better Performance per Dollar, Rapid Externalization of TPU stack, Growing Customer Base, Ironwood, TPUv8i, Reducing CUDA Moat 

For more than a decade, the industry has watched Google build an empire on its own silicon. Search, Ads, YouTube, and every generation of Gemini run on TPUs. Few accelerators have attracted as much architectural scrutiny or as much debate about what their performance and economics would look like outside the company that designed them. Anthropic being the biggest user of TPUs, surpassing Deepmind’s own use by 2029.

Google’s internal success was never the question. The question was how much of that advantage the rest of the industry could actually get. Could you take an open-weight model, serve it through a familiar inference engine, and beat NVIDIA on the economics that matter to your business?

Today, we are publishing the first third-party inference results for TPUv7 Ironwood on InferenceX Official Preview. In our apples-to-apples comparisons against B200/B300, Ironwood delivers up to 50% better performance per dollar. Its advantage extends across much of the Pareto curve, and we examine the economics from both sides: Google’s internal total cost of ownership and the external TCO an actual customer pays.

Ironwood (TPUv7) is the first generation in which Google is competing for others’ inference workloads with chips that can be purchased outright or rented through its own cloud. In November 2025, we already said that Anthropic loves TPUs and committed to over one million of them (around 400k+ in direct purchases and 600k+ rented through GCP), used mainly for training but also for inference. Our Accelerator Model has the latest figures for Anthropic’s TPU shipments and Google’s overall TPU shipments by quarter, plus estimates for TPUv8i, v8t, and various v9 / v10, and more

We are excited by how quickly the new TorchTPU stack is developing, the external stack for TPUs. Later in the article, we will discuss the upcoming work needed for TPU software externalization, including optimizing speculative decoding, disaggregated prefill, KV-cache offloading, multi-turn agentic workloads, and more. Even so, we at SemiAnalysis strongly believe that TPU externalization is heading in the right direction and moving full steam ahead. Furthermore, unlike AMD, which is still learning how to build a test-first software culture, Google has decades of software engineering experience and an extremely well established quality-driven culture, so we expect external TPU software to mature rapidly.

In this article, we will cover all the optimizations that went into TPU kernels and the serving stack for open-weight models, including DP attention optimization, MoE routing and kernel optimization, and reducing padding in GDN kernels. We will also take a deep dive into the TPU system and discuss the next steps the amazing TPU performance engineers are pursuing to make the stack widely available.

Google has spent more than a decade demonstrating what it can build with TPUs. Now we get to measure what the rest of the industry can do with them.

Shoutout to the Google (Chris Chan, Jahangir Hasan, Wangyuan Zhang, Anne Stern, Puneith Kaul, Ruizi Dong, Sangam Jindal, Qi Zhou, Madhan Jaganathan, Gang Ji, Jun Wan, Devanshu Jain, Jiaxin Cao, Srinath Mandalapu, Haowen Ning) and Inferact teams and RedHat Teams (Michael Goin) for this amazing TPU foundation and performance! Furthermore, shoutout to the RadixArk team that is also working on TorchTPU SGLang.

InferenceX Official Preview: TPUv7 Ironwood vs. Blackwell and Blackwell Ultra 

We are already seeing strong results from the upcoming native TorchTPU vLLM stack in apples-to-apples comparisons against Nvidia GPUs. Google is using Qwen3.5 397B in FP8 as the initial bring-up model. Later sections take a deep dive into why the new TorchTPU approach is a marked improvement over the previous TorchAX path for external TPU vLLM/SGLang serving.

Once that foundation is in place, Google plans to extend support to other open-weight models, including Kimi K3 and GLM5.3. Once a handful of models are well optimized, we believe adding optimized support for a wide range of popular open models near day 0 becomes far easier. Today, vLLM and SGLang concentrate their day-0 support on Nvidia, with passable day-0 coverage for AMD. We expect TorchTPU to be stable enough in the near future that vLLM and SGLang maintainers may add TPUs to that day-0 list. The stack is expected to leave private beta & be open sourced around October. 

In apples-to-apples comparisons of aggregated serving with FP8 and single-token prediction, we are seeing up to 50% better performance per dollar from TPU than from B200 and B300 running FP8 in aggregated serving. When serving models using FP4 on NVIDIA GPUs, there is quality loss verus FP8. TPUv7 does not have native FP4 computation thus on FP4, NVIDIA GPUs still maintains the lead. This will change with TPUv8i which has native FP4 support thus we strongly believe TPUv8i Boardfly will be competitive to Rubin NVL72.... 

....MUCH MORE
*
April 2017 - Watch Out NVIDIA: "Google Details Tensor Chip Powers" (GOOG; NVDA)
We've said NVIDIA probably has a couple year head start but this bears watching, so to speak....

And many, many more, including November 2025 "CHIPS: Google's Tensor Processing Unit (Finally) A Viable Competitor For Nvidia (GOOG; NVDA)" 

Sunday, August 9, 2026

SemiAnalysis Looks At SpaceX (SPCX; MSFT; NVDA)

Talk about your "big if true"/"big if, true". The numbers are just so mindbending. 

From SemiAnalysis, August 7:

SpaceX 10GW in 2027 – Why It’s Real, Will Drive $300B ARR for SpaceX, and Why Microsoft Will Be the Largest Offtaker
Inference at 100B/GW/year, SpaceX's stellar pace, Microsoft's 10GW 2026 Awakening, Azure Can Grow Triple-Digits

Elon Musk shocked the world, once again, when he announced on SpaceX’s first earnings his Gigawatt ambitions for next year. He “conservatively” aims to build & deliver an incremental 6-8GW in 2027 alone, with potential for that number to be well above +10GW. At 50B per GW, that’s $300-500B in capex in 2027, on par with what we expect from AWS and Google – an unbelievable number for a company significantly less profitable than rival hyperscalers.

Yet, we believe that the number is real. We see SpaceX on track to build about 10GW by year-end 2027. We’ve evaluated all sites suitable for SpaceX and provided the list to our Datacenter Model subscribers. Our Energy Model subscribers also have the precise list of gas generation equipment available, quarter by quarter, by 30+ turbine, engine, fuel cell suppliers. We provided much of this data, before the market woke up to it. Below, we discuss how Elon bypasses typical datacenter construction constraints.

As explained in our Meta Compute deep dive, large-scale + near-term compute is a remarkably scarce combination, and it’s priced at a huge premium – up to $50B/GW/year. However, AI labs can handle it and make a good living off it.

Our Tokenomics Model and our Inference Simulator demonstrate that at realistic performance levels (e.g. tokens/sec per GPU), both OpenAI and Anthropic can generate over $100B/GW/year of revenue when selling API inference on a GB300 cluster. This is significantly more than the costs of renting a GB300 cluster for a year at current neocloud prices.

Serving inference tokens is unbelievably profitable for the frontier model companies.

 

Source: SemiAnalysis Tokenomics Model, SemiAnalysis Inference Simulator

We assume around $12B/GW/year of cost per year, using a conservative rental pricing rate of $3/GPU-hr, and make a token production estimate using our Inference Simulator with a frontier-class model architecture and our agentic coding benchmark, AgentX (part of InferenceX), which is built by collecting real production coding traces. We blend that token production rate between input, cache-read, cache-write, and output token costs at our real workload ratios, and produce the final estimate, exceeding $100B/GW/year.

For background, our Inference Simulator is built from the ground up with a fundamental understanding of how modern AI accelerators work. We build a roofline and realistic performance model for how frontier models work during inference, with timings for every operation and a real trace output. It is an end-to-end simulation of the actual workload executing on the actual silicon. We have validated the simulators fidelity on a wide range of accelerators and workloads and continue to improve its ability to accurately forecast performance of future accelerators based on design specifications....

***** 

...Please reach out to sales@semianalysis.com for more information on how we apply the Inference Simulator for custom research and analysis.

Beyond OpenAI and Anthropic, there is actually a third company in the world capable of printing such economics per GW: Microsoft. Having full access to OpenAI models, they can generate the exact same revenue and margin per MW, while paying none of the training costs. Satya nailed the negotiations with OpenAI: the deal reworked in April 2026 dropped the old 20% revenue share from the equation. Put simply, Microsoft has a giant incentive to procure as many MWs as possible, as fast as possible. While much of their datacenter capacity currently goes to OpenAI at ~14M/MW/year, they have the opportunity to improve that mix. The potential impact is Microsoft Azure accelerating revenue growth from ~42% to over 100% by next year. A once-in-a-generation opportunity, that SpaceX is incredibly well positioned to serve.

 

Source: SemiAnalysis Tokenomics Model

While Microsoft signing 3GW with SpaceX for 50B/GW/year sounds insane, we view it as possible for two reasons:

  • 1/ Microsoft is already preparing for an epic datacenter ramp. As discussed below, they’ve signed 10GW of contracts year-to-date, for over $300B of total contract value (not including the GPU cost). We expect much more to be signed. Caveat: these contracts contribute to late 2027 and 2028 capacity. There is a near-term gap to fill.

  • 2/ With a 90-day cancellation policy, akin to the SpaceX deals with Anthropic and Google, there is zero balance sheet risk. This is remarkably easy for Amy Hood to sign off, given the revenue opportunity.

For SpaceX, the next natural question is financing. How can Elon afford to pay so much CapEx without the balance sheet of the leading hyperscalers? We expect a combination of the two following items:

  • 1/ Support from Nvidia, in the form of vendor financing to lower the upfront cash cost. This is likely why Elon declared to be Nvidia exclusive on the earnings call! As our Accelerator Model has repeatedly explained, xAI/SpaceX have actively evaluated alternatives like TPU and AMD – so the financial argument likely made them abandon these and focus on Nvidia.

  • 2/ Operating cash-flow financing led by industry-high pricing, enabled by fastest timelines: SpaceX will continue to sell large-scale compute with 3-5 months lead time, an unbeatable offering, and price it accordingly at 30-50M/MW/year. That pays back the capex in less than a year. We dived into this in our Meta Compute article.

The implications of this are a path to $300B of ARR by the end of 2027 for SpaceX. This assumes only 50% of their 2027 incremental compute is monetized, the reminder being for the Grok & Cursor teams for training (no inference revenue modelled)....

....MUCH MORE 

We've chronicled much of the Elon - Jensen frenemy relationship in real-time for over a decade. On August 4 Musk said SPCX would use NVDA's platforms exclusively.

It wasn't always apparent that this is how things would turn out but the two centi-billionaires seem to get along. From a July 2023 post, "CORRECTED—Earnings - Tesla Reports, Stock Slides, Elon's Buying A Supercomputer (TSLA)":

....For some background on Tesla and AI here is our introduction to June 9's "Elon Musk Predicts Nvidia’s Monopoly in A.I. Chips Won’t Last" (NVDA; TSLA)":

Before we get to the headline story, some background. Tesla and Nvidia have a history.

In 2015 - 2016 when everyone thought that autonomous driving was just around the corner, the challenge was seen as both a sensor issue, for example: LIDAR vs cameras, and a machine learning/artificial intelligence problem which boils down to training the AI 'puters with as much data as you can so that out in the real world the autonomous vehicle can say to itself: "Yeah, I've seen this situation before, here's the response that worked best. Both the training and the on-the-road-recall, if they are to be anywhere near efficient, require the fastest chips you can find. Tesla had a whole bunch of data from a few billion miles of actual driving for computers to train on, and, combined with Nvidia's fastest-in-the-world GPU chips, it was a match made in heaven.

Except it wasn't.

The challenge of autonomous driving on open roads alongside non-autonomous vehicles was bigger than anyone in that simple, optimistic time ever envisioned, even in their nightmares. Here's one example about Waymo from a 2017 post:

"When Google was training its self-driving car on the streets of Mountain View, California, the car rounded a corner and  encountered a woman in a wheelchair, waving a broom, chasing a duck. The car hadn’t encountered this before so it stopped and waited."

In May 2015 we were posting " Nvidia Wants to Be the Brains Of Your Autonomous Car (NVDA)" and seven months later the more declarative "Class Act: Nvidia Will Be The Brains Of Your Autonomous Car (NVDA)"

Then in October 2016, what was probably the high-water mark for the relationship "Nvidia Could Make $1B From Tesla's Self-Driving Decree: Analyst (TSLA, NVDA)"

Sadly, the task was just too difficult but Mr. Musk thought it was doable if only he could get even faster chips than Nvidia had on offer:

NVIDIA Partner Tesla Reportedly Developing Chip With AMD (TSLA; NVDA; AMD) 
Today in leveraged WTFs....

"During a talk at a private party, Elon Musk said Tesla is developing specialized AI hardware "'That we think will be the best in the world;" (TSLA)  

"Tesla says it’s dumping Nvidia chips for a homebrew alternative" (TSLA)
The only reason for Tesla to do this is that NVIDIA's chips are general purpose whereas specialized chips are making inroads in stuff like crypto mining (ASICs), Google's Tensor Processing Units (TPUs) for machine learning and Facebook's hardware efforts. 
 
 Watch Out NVIDIA: "Google Details Tensor Chip Powers" (GOOG; NVDA)
We've said NVIDIA probably has a couple year head start but this bears watching, so to speak....

Culminating in August 2018's
"Nvidia CEO is 'more than happy to help' if Tesla's A.I. chip doesn't pan out" (NVDA; TSLA)

And now on to the headliner, from Observer, June 8:
Elon Musk Predicts Nvidia’s Monopoly in A.I. Chips Won’t Last....

And possibly related July 13:
Elon Musk's x.AI Launches
The company was formed in March so it's valuation is probably around a hundred billion or so.

Just kidding. I have no idea what sort of valuation it has been assigned. x.AI is a Nevada corporation which, as our corporate attorney readers well know, is handy as hell for a privately-held stealth company. As part of the company's coming-out I think they dropped the period in the name on the original incorporation papers.

Mr. Musk was one of the founder/funders ($100 million gift not equity) of ChatGPT parent OpenAI when it was a .org (non-profit) and seemed a bit miffed when Sam Alman hooked up with Microsoft to the tune of $10 billion.

So Elon went out and bought a garage-full of GPUs.

Here's a twofer, first up TechCrunch, July 12:....

Wednesday, March 18, 2026

Memory: Shortage Could Last Five Years, It's The Wafers

One of the first sources to relay the concern that the memory chip shortage would not be over quickly was Tom's Hardware in October 2025, linked here on the blog on January 3:

"AI data centers are swallowing the world's memory and storage supply, setting the stage for a pricing apocalypse that could last a decade" 

Now we return to Tom's to see what the head honcho at #2 memory chip maker SK Hynix has to say, March 18:

SK Group chairman says memory chip shortage will last until 2030 — wafer supply trails demand by 20% 

SK Hynix's CEO is expected to announce price stabilization measures soon. 

SK Group chairman Chey Tae-won told reporters at Nvidia's GTC conference in San Jose on Monday that the global memory chip shortage is likely to persist for another four to five years, with industry-wide wafer supply lagging demand by more than 20%, Bloomberg reported. Chey, whose conglomerate controls SK Hynix, said leading memory makers are expanding capacity but are unlikely to fully meet demand until around 2030 because securing additional wafers takes at least four to five years, according to The Korea Times. 

SK Hynix holds roughly 57% of the global HBM market and 32% of overall DRAM, and the company is currently building a $13 billion HBM packaging and testing facility at its Cheongju complex in South Korea, with construction scheduled to begin next month and completion targeted for the end of 2027.

Article continues below...

....MUCH MORE  

And it's not just silicon wafers for memory chips.

From SemiAnalysis, March 12:

The Great AI Silicon Shortage
TSMC N3 Wafer Shortages, Memory Constraints, Datacenter Bottlenecks, Supply Chain Wars Winner 

Token demand is skyrocketing and the need for AI compute continues to accelerate. The improvement in model capabilities combined with the rapid emergence of agentic workflows has driven a surge in user adoption and aggregate token demand. Anthropic added a staggering $6B of ARR in the single month of February alone driven by broad adoption of agentic coding platform Claude Code, and if Anthropic had more compute they would have added more. Despite a huge AI infrastructure buildout over the past few years, available compute is scarce. On-demand GPU prices continue to go up even for Hoppers which are almost 2 generations old.

From our own experiences, we have reached out to every neocloud we know asking if they have small clusters available, but everything is already firmly locked up. This tight supply environment explains the sharp reset in hyperscaler capex plans. Consensus estimates have moved materially higher across the board, with Google standing out as the most extreme example, where 2026 capex expectations have roughly doubled versus prior expectations, primarily driven by datacenter and server spend.

 

Source: Company Earnings, Bloomberg

This is a tremendous level of spending, and hyperscalers would deploy even more capital if they could, but they are constrained by one critical factor: silicon supply. There is simply not enough advanced logic and memory fabrication capacity to support the pace of compute deployments. While the AD (After Da launch of ChatGPT) era has been riddled with various constraints such as CoWoS packaging and datacenter power, we are now firmly in the silicon shortage phase.

 

Source: SemiAnalysis Accelerator Model

The TSMC N3 Shortage

One of, if not the, biggest constraints is TSMC’s N3 logic wafer capacity. TSMC’s N3 family started shipping for revenue in 2023, with demand initially driven primarily by smartphones and PCs. N3 got off to a shaky start, with the first variant “N3B” having yield issues and being too expensive relative to the density improvement. Greater adoption came with the refined N3E process, a relaxed variant with far fewer EUV layers and therefore lower cost. Key smartphone and PC customers include Apple, which uses N3 variants for its M3 to M5 Mac chips and A17 to A19 iPhone processors, Qualcomm for its Snapdragon 8 Elite series, MediaTek for its Dimensity smartphone SoCs as well as select automotive and PC chips, and Intel for its Lunar Lake and Arrow Lake client processors.

 

Source: SemiAnalysis Foundry Model

Up until today, N3 demand has been driven primarily by consumer electronics. In 2026, all the main AI accelerator families are transitioning to N3, and AI will account for the majority of N3 demand before transitioning to N2 and beyond.

We can see in the table below the industry-wide convergence toward TSMC’s N3 family as the leading process node for AI accelerators heading into 2026. NVIDIA transitions from 4NP with Blackwell to 3NP with Rubin. AMD, typically the earlier adopter of new nodes, has already adopted N3 for MI350X and will stay on N3 for the AID and MID tiles for MI400 (XCD is N2). Google’s TPU roadmap shifts fully to N3E starting with TPU v7, with TPU seeing a huge upsize in program volumes this year. AWS also transitions to N3P with Trainium3. Meta’s MTIA follows a similar path, though it will be at much lower volumes.

 

Source: SemiAnalysis Accelerator Model

This shift is not limited to XPU silicon. The Vera CPU used in VR racks uses N3P for all its silicon. There is also networking silicon in the form of the NVLink 6 switch, as well as scale out switches like Tomahawk 6 and Spectrum 6. With Rubin offering 1.6T of scale out network per GPU, Rubin kicks off the adoption of 3nm 200G optical DSPs.

This sudden convergence of N3 adoption coupled with the continued growth of AI compute demand has resulted in a huge demand shock for N3 wafer capacity. TSMC has been caught flat-footed, with wafer capacity expansion failing to keep pace with surging AI demand. How did this happen? Although the greatest compute buildout in history began in late 2022, TSMC’s capex only exceeded its previous peak in 2025. This year, TSMC is going to smash through last year’s record Capex, because they have realized how far customer demand is exceeding their capacity....

....MUCH MORE 

It's a pretty big deal. If interested see:

Memory: "The inflation spark that could become a deflation shock?"

From M&G's Bond Vigilantes, February 27:
Memory chips have quietly become the most important commodities in the global economy.

 “'Entry-Level PC Segment Will Disappear by 2028,' Says Gartner, as Soaring Memory Costs Start to Cripple Manufacturers"

"....Micron, SanDisk & Memory Stocks Are Crashing Today"

 Thanks for the Memories: "South Korea’s Kospi plunges 12% amid broader declines in Asia markets as Iran conflict rages"
The index, which has been driven by the memory chip makers, Samsung Electronics Co. and SK Hynix Inc. et al., up over 145% from March 2025 to the February 25, 2026 peak is now down 10% on the day, March 4th....

Sunday, February 15, 2026

"Google's year in review: 8 areas with research breakthroughs in 2025" (GOOG)

This was posted on December 23, 2025 so some of the points made have already been superseded by events but it is a good overview of what Google wishes to share/highlight.

From the Google blog, The Keyword:

This was a year of AI agents, reasoning and scientific discovery. 

Delivering breakthroughs on world-class models
This year, we significantly advanced our model capabilities with breakthroughs on reasoning, multimodal understanding, model efficiency, and generative capabilities, beginning with the release of Gemini 2.5 in March and culminating in the November launch of Gemini 3 and the December launch of Gemini 3 Flash.

Built on a foundation of state-of-the-art reasoning, Gemini 3 Pro is our most powerful model to date, designed to help you bring any idea to life. It topped the LMArena Leaderboard and redefined multimodal reasoning with breakthrough scores on benchmarks like Humanity’s Last Exam — a fiendishly hard test for AI models to see if AI can truly think and reason like humans — and GPQA Diamond. It also set a new standard for frontier models in mathematics, achieving a new state-of-the-art of 23.4% on MathArena Apex. We followed shortly with Gemini 3 Flash, which combines Gemini 3's Pro-grade reasoning with Flash-level latency, efficiency and cost, making it the most performant model for its size. Gemini 3 Flash's quality surpasses our previous Gemini 2.5 Pro-scale model's capabilities at a fraction of the price and substantially better latency, continuing our Gemini-era trend of 'the next generation's Flash model is better than the previous generation's Pro model'.

Learn more about our progress on our world-class AI models this year:

Gemini 3 Flash price & benchmark table. 

....MUCH MORE 

Friday, February 13, 2026

"Nvidia Shares Go Cold Even as Big Tech Spending on AI Balloons" (NVDA)

This is true.

It is also true that come October - November, after the boys of summer have gone, Nvidia will still be with us. It is sometimes difficult to comprehend just how truly awesome the company is, technologically or financially or just about any dimension you care to examine.

As just one example, after a decade of watching Google and their Tensor Processing Unit slowly catch up to, and then surpass, Nvidia's GPU's on the inference side of things, Nvidia did a $20 billion acquihire of Groq's wunderkinder CEO and licensed their inference technology.

For most any other company on earth that would be a tectonic move. For Nvidia it was just another step on the journey. And truth be told,  Groq's CEO isn't really a kid any more. While he was at Google he invented the TPU. 

From Bloomberg via Advisor Perspectives, February 13: 

Big Tech keeps raising its spending plans for artificial intelligence infrastructure, yet shares of Nvidia Corp., one of the biggest beneficiaries of that flood of cash, have been largely stagnant for months.

The stock is up less than 1% since the beginning of the fourth quarter and has been largely range bound despite hitting a record high in late October. It’s also barely beating the S&P 500 Index to start 2026, a slowdown from Nvidia’s nearly 40% leap in 2025 following two consecutive years of triple-digit percentage gains.

Even ballooning capital spending from Meta Platforms Inc., Alphabet Inc., Microsoft Corp. and Amazon.com Inc. — estimated to exceed $600 billion in 2026 — hasn’t been enough to meaningfully boost the stock amid increasing anxieties about returns on those investments.

https://www.advisorperspectives.com/images/content_image/data/5f/5fc06395a11f9435723c4be6d4384a3d.jpg 

“There is perhaps growing concern that the ultimate revenue from AI will simply not keep up with the capex spend that’s been announced,” said JoAnne Feeney at Advisors Capital Management, adding that more spending now raises the probability that the market will reach satiation faster. It’s “going to move up the date at which they pause and let the new compute be digested.”

The cyclical nature of the chip industry is baked into Nvidia’s valuation, which has compressed as revenue growth is expected to slow in the coming years. Sales are projected to expand 58% in the current calendar year and 28% in 2027, according to data compiled by Bloomberg.

Nvidia shares trade around 24 times profit estimates, roughly in-line with the Nasdaq 100 index and a slight premium to the S&P 500. Even though this price-to-earnings ratio is far below the five-year average for the stock at 38 times, investors aren’t counting it as a discount....

....MUCH MORE 

$183.03 down $3.91 (-2.09) last. 

Our judgement: If you want to own the future, own this company. That is not a new opinion.

Wednesday, January 28, 2026

So it Begins: "Silicon Valley Wants to Build A.I. That Can Improve A.I. on Its Own"

The headline at TechCrunch was "AI chip startup Ricursive hits $4B valuation two months after launch" 

Serious money believes these women are on to something. 

From the New York Times, January 26:

Ricursive Intelligence, founded by two former Google researchers and valued at $4 billion, is among several efforts to automate the creation of artificial intelligence. 

One new start-up is called Recursive with an “e.” Another is called Ricursive with an “i.” They are trying to do the same thing: Build artificial intelligence that can improve itself without the help of humans, an obsession of Silicon Valley technologists for decades.

Ricursive Intelligence, based in Palo Alto, Calif., is working with the specialized computer chips that power today’s chatbots. Founded by two former Google researchers, Anna Goldie and Azalia Mirhoseini, Ricursive aims to build A.I. systems that can improve the design of these enormously complex chips.

If A.I. systems can produce better chips, they argue, the chips will produce better A.I. systems. And then the process would repeat on and on as technology got better and better.

“The idea of a recursive self-improvement loop is what inspires us,” said Dr. Goldie, who did similar work with Dr. Mirhoseini at Google.

Ricursive has raised $335 million from venture capital firms like Sequoia, Radical Ventures, Lightspeed and DST Global. Although it is less than a year old and has fewer than 10 employees, it is valued at $4 billion.

The company is among several new A.I. start-ups that have raised enormous amounts of money in recent months. Just last week, Humans&, founded in San Francisco by former researchers at labs like Anthropic and Elon Musk’s xAI, raised $480 million. It is just three months old and valued at $4.48 billion.

Even as many financial analysts and industry insiders warn of an A.I. bubble, vast amounts of money continue to pour into the field. This is partly because the raw computing power needed to build A.I. technologies is so expensive. If investors want to bet on a new idea, hundreds of millions of dollars are increasingly the table stakes to get into the A.I. game.

Recursion is a term commonly used by mathematicians and computer programmers. It refers to a mathematical function or procedure that feeds itself. After a procedure generates some information, it uses that information to generate something else. That process can go on forever.

That mathematical idea has inspired A.I. researchers for decades. Rather than building just a mathematical function that feeds itself, they aim to build an A.I. system that feeds itself.

In 2017, as the latest wave of A.I. development first gained steam, Google built technology called AutoML. ML was short for “machine learning,” referring to computer algorithms that learn skills by analyzing data. With AutoML, Google took this idea a step further. It built a machine-learning algorithm that learned to build other machine-learning algorithms.

At OpenAI, the maker of ChatGPT, researchers are building what they call an “automated A.I. researcher.” By the fall, they hope to have a system that can do the work of a less experienced researcher, before steadily improving on the technology, the company’s chief executive, Sam Altman, said.

That is similar to the aim of another new start-up, Recursive AI, founded by Richard Socher, who oversaw A.I. research at the cloud computing giant Salesforce. His start-up has yet to publicly announce itself, but its mission is already a topic of discussion across Silicon Valley’s tightknit community of A.I. researchers. Recursive AI is also valued at $4 billion, according to a person familiar with its latest funding round who spoke on the condition of anonymity. The news was first reported by Bloomberg.

Although technologies as far back as Google’s AutoML have shown that A.I. can help improve A.I., these efforts are still a very long way from a future where humans can be removed from the process, said Div Garg, chief executive of AGI, a San Francisco start-up that is working to build increasingly intelligent computer technologies.

“They work well for very specific tasks,” he said.

At Google, Dr. Goldie and Dr. Mirhoseini built A.I. technology that could improve the design of the company’s in-house computer chip. Called the tensor processing unit, or TPU, the chip was designed for building and running A.I. technologies....

....MORE 

Monday, January 26, 2026

"Microsoft unveils Maia 200 AI chip, claiming performance edge over Amazon and Google" (MSFT; AMZN; GOOG; NVDA)

From GeekWire (Seattle, Redmond, Kirkland), January 26: 

Microsoft on Monday announced Maia 200, the second generation of its custom AI chip, claiming it’s the most powerful first-party silicon from any major cloud provider. 

The company says Maia 200 delivers three times the performance of Amazon’s latest Trainium chip on certain benchmarks, and exceeds Google’s most recent tensor processing unit (TPU) on others.

The chip is already running workloads at Microsoft’s data center near Des Moines, Iowa. Microsoft says Maia 200 is powering OpenAI’s GPT-5.2 models, Microsoft 365 Copilot, and internal projects from its Superintelligence team. A second deployment at a data center near Phoenix is planned next.

It’s part of the larger trend among cloud giants to build their own custom silicon for AI rather than rely solely on Nvidia. Google has been refining its TPUs for nearly a decade, and Amazon’s Trainium line is now in its third generation, with a fourth already announced. 

Microsoft first revealed its custom chip ambitions in late 2023, when it unveiled Maia 100 at its Ignite conference. Despite entering the race late, Microsoft makes the case that its tight integration between chips, AI models, and applications like Copilot gives it an edge.... 

....MUCH MORE 

Here's MSFT -  Deep dive into the Maia 200 architecture

Microsoft is up 1.6% on the news, Nvidia is down 0.6%. I don't know if it's on the news or ennui. 

Saturday, December 27, 2025

"NVIDIA’s Christmas Eve 'Hackquisition' Miracle" (NVDA; GOOG)

From Spyglass, December 25:

NVIDIA gets a creator of the TPU, access to inference tech & IP, and stops anyone else from getting such things. All for a mere $20B...  

Twas the night before Christmas and all through the house, not a creature was stirring, not even a... wait. What's that? Is that Jensen Huang scurrying around doing deals over the holidays?!

It sure is!

Well, presumably he had been working on the deal with the specialized AI chip startup Groq before the holidays started. But I also wouldn't rule out that this deal came together in a hurry given what we know about how he's operated in the past with some mega deals, such as the um, (up to) $100B investment in OpenAI. A deal which came together quickly, perhaps on a trip with a certain President, and perhaps right after the rumors of OpenAI potentially looking at a deal with Google to use their TPUs – we'll come back to that. It was also a deal which was announced three months ago but apparently still has not been finalized, by the way, so there's that.

Anyway, you can't help but wonder if the real key for the timing here may have been to bury it under the spirit of Christmas. Because boy is it a doozy.

CNBC first broke the news with the headline: "Nvidia buying AI chip startup Groq for about $20 billion in its largest acquisition on record". But as it turns out, that wasn't quite right. And actually, it was NVIDIA that wanted to make that clear. They weren't buying Groq, they were merely paying Groq that $20B for a "non-exclusive licensing agreement".

Oh, you've heard that terminology before? Of course you have. It's the magical incantation one must cite in order to summon the spirits of the "hackquisition".

With this deal, NVIDIA is following their Big Tech brethren down this now well-trodden path. Microsoft, Amazon, Google, Meta, and NVIDIA have done deals which aren't technically acquisitions, but effectively are. Because they allow the acquiring – sorry, non-acquiring – company to sort of pick and choose what they want from the acquired – sorry, non-acquired – company in exchange for considerations that can go to... well, sort of anyone they choose. Certainly the investors to get them to sign off on such deals, and often the key employees at those companies. Sometimes the other employees of the companies too, but that's mainly so they don't feel bad and/or raise a stink about the faux-deal. Because when they do that, all hell tends to break loose, at least from a PR-perspective.

Anyway, the first wave of "hackquisitions" were almost more like "hackquihires" because they were simply a way to get access to key talent at those companies and any licensing agreements were seemingly an excuse to make it all a little less blatant. But as such deals have grown in size, they've grown to look even more like actual acquisitions. For example, Meta's nearly-$15B deal with Scale.ai bought them 49% of that company alongside their key talent, namely CEO Alexandr Wang. Why? Presumably Meta wanted more for that amount of money and they thought Scale, a hot company at the time, might still give them some upside. Of course, they were also effectively gutting said company and it seems decidedly less hot now – especially since many of their customers were Meta competitors who no longer wanted to send their data to Scale – shocking, I know. Still, Meta also likely believed they could use Scale's service to help with their future model training – efforts which were rebooting alongside bringing Wang on board.

NVIDIA's Groq deal seemingly has similarities. Namely, the key was clearly to bring CEO Jonathan Ross on board. And every story also points to company president Sunny Madra being crucial as well. But beyond that, it sounds like NVIDIA may actually care about some of the IP rights here (which they're presumably getting a license to with their "non-exclusive licensing agreement"), to be able to leverage Groq's techniques in would-be future chips.

In that way, "non-exclusive" feels less important here – that's another framing to make it seem less like an acquisition, but is anyone else really getting access to this IP now? Regardless, NVIDIA probably feels confident that with Ross and Madra – not to mention their own in-house prowess – they'll be able to implement it and execute upon it far better than anyone else. And they're undoubtedly not wrong.

And that points to another layer to this as well. Ross is not just a co-founder of Groq, he's also the creator of the TPU – something which he cites in his own bio. You may recall the TPU was last a part of a major news cycle when Jensen Huang was "delighted" about Google's success with their AI chips. How do I know that NVIDIA is actually not so "delighted" about Google's success here and is in fact sweating the rise of the TPU? Well, this deal, for one!....

....MUCH MORE 

Recently:

And on the TPU:

November 25 - CHIPS: Google's Tensor Processing Unit (Finally) A Viable Competitor For Nvidia (GOOG; NVDA)

....While we've been tracking the GOOG's TPU progress for a decade, this morning Zerohedge found a dandy little write-up on the chips that we'll go with rather than use back-links.... 

Friday, December 26, 2025

"Nvidia Licenses Groq's AI Chip Tech, Grabs Top Execs In Not-Quite Takeover" (NVDA)

From Investor's Business Daily, December 25:

Nvidia (NVDA) will license key technology from AI chip maker Groq and bring over key personnel from the startup, Groq announced Wednesday afternoon. That followed a report that the Dow tech giant would pay $20 billion for Groq. Nvidia stock was indicated fractionally higher Thursday night.

Groq has entered into a non-exclusive licensing deal with Nvidia for the startup's inference technology, the company said in a press release. As part of the agreement, Groq founder and CEO Jonathan Ross, President Sunny Madra and some other Groq employees will join Nvidia.

Groq will remain an independent company, with CFO Simon Edwards becoming CEO. Specifically, Groq's cloud business is not part of the deal. But most assets may be going to Nvidia.

Earlier, CNBC had reported that Nvidia would pay about $20 billion to buy Groq outright.

Financial terms of Nvidia's licensing and hiring deal were not given. Nvidia itself has said nothing as of Christmas morning.

There is some speculation that the not-quite takeover structure is an effort to avoid antitrust objections.

Media reports still are touting a $20 billion figure, though if true it's unclear if that would be a lump sum or include some future milestone payments.

Groq Founded By Google TPU Creators....

....MORE 

We had this story in the link-vault before the Groq story came out, it is still accurate. From CNBC December 4: 

Nvidia has a cash problem — too much of it

In early pre-market trading NVDA is up $1.08 (+0.57%) at $189.69  

Monday, December 22, 2025

A.I.—Google's Vertical Integration Is A Real Advantage (GOOG)

The GOOG is down a bit today, to $308.19 and like many other issues is trading just under the all-time-high from earlier this year.

From Barchart, November 25:

Google is Winning the AI Race by Stealing Exxon's Business Model. Here's How 

Alphabet (GOOG) (GOOGL) has been on a tear since the favorable court decision about monopolistic advantages. The chart below shows the percentage returns in Google stock compared to the other Magnificent 7 names – Apple (AAPL), Amazon (AMZN), Meta (META), Nvidia (NVDA), Microsoft (MSFT), and Tesla (TSLA) – over the past year. 

 

www.barchart.com

There’s more going on here than just a court victory – and I mean besides the obvious point that Google is the only Mag 7 stock that is truly beating the tape.

The answer? Google appears to have won the finish line to artificial intelligence (AI) monetization. Not only that, they’ve won by a wide margin, and stock market capital is flowing out of the other hyperscalers to the victor.

Gemini Helps Blaze a Path to AI Victory 
Google’s Gemini gained 8% user growth in the most recent quarter, with web traffic to the chatbot surging in September after the launch of new AI image generation tools. Rave reviews for Gemini 3, including from Salesforce (CRM) CEO Marc Benioff, have also pushed the stock higher this month.

Google's tensor processing unit (TPU) demand is exploding as Gemini gains wide acceptance. More adept than traditional GPUs at handling complex AI workloads, and with a longer life cycle, Google delivers 50% productivity improvement with TPU than what is offered using NVDA hardware. As a result, developers are switching to Gemini midstream at an increased cost to substitute.

Plus, Google is already generating income across all spectrums, while relative newcomers like OpenAI (parent company of Gemini rival ChatGPT) are still burning through billions of cash.

Is Google the 'Big Oil' of AI? 
In an industry where everyone is renting from each other, Google designs its chip; owns the data centers; runs the software; manages the pricing; manages the margins; and handles distribution.....

....MORE 

Friday, December 19, 2025

"The Trillion-Dollar Race to Fragment the Nvidia Monopoly" (NVDA)

Am overview but an overview with insight. 

From EE Times, December 9: 

Nvidia’s ongoing fight to maintain technological and market dominance in AI inference. 

For the past ten years, Nvidia has dominated the market for advanced computer chips used in machine learning and artificial intelligence.

Driven by its proprietary CUDA software and rapid innovation, Nvidia became synonymous with AI processors, briefly reaching a $5 trillion market value this year. Between February and October 2025, Nvidia reported $147.8 billion in sales from chips, network connections, and related hardware supporting AI growth.

Even though Nvidia’s newest and most powerful processors, the Grace Blackwell series, are selling out quickly, its dominance is starting to weaken. Instead of facing just one competitor, Nvidia is now challenged on many fronts as the industry shifts toward more specialized hardware.

Nvidia’s strong sales and high profit margins are driven by scarcity, primarily due to manufacturing limits. The main bottleneck for high-end chips is the limited capacity for advanced Chip-on-Wafer-on-Substrate (CoWoS) packaging at TSMC.

Nvidia consumes most of this limited production, but competition for chip supply is intensifying. TSMC, the sole manufacturer, plans to expand capacity to 100,000 wafers per month by 2026. As supply constraints ease, companies like Google and AMD are expected to benefit.

Nvidia now faces significant risk as the industry shifts from experimenting with large foundation models to prioritizing large-scale, cost-effective inference.

Major cloud providers are moving away from reliance on Nvidia’s CUDA ecosystem. They are investing in their own specialized chips for high-volume inference, where operating costs now exceed training costs.

Nvidia’s largest customers become competitors

The “Big Four” North American hyperscalers—Google, Amazon Web Services, Microsoft, and Meta—account for the largest capital expenditures in this sector. Their collective shift to custom silicon is a strategic move to ensure competitiveness.

Google, part of Alphabet, initiated the move toward custom AI chips with its Tensor Processing Units (TPUs). The latest version, TPU v7 ‘Ironwood,’ is optimized for inference. Ironwood features a large shared memory, enabling up to 9,216 chips to connect in a single ‘superpod,’ which addresses memory constraints in large Mixture-of-Experts (MoE) models.


Meta Platforms may lease and potentially purchase Google’s TPU chips for its data centers starting in 2027. This would mark a shift for Google, positioning it as a merchant chip supplier. Some estimates suggest Google could capture up to 10% of Nvidia’s annual revenue, amounting to billions of dollars.

Amazon focuses on training and cost efficiency

Amazon Web Services (AWS) is pursuing improved price-performance to attract businesses seeking alternatives to Nvidia’s expensive chips. AWS claims its Trainium chips can reduce training costs by up to 50% compared to GPUs, targeting mid-range AI workloads....

....MUCH MORE 

Friday, November 28, 2025

"Robots and AI Are Already Remaking the Chinese Economy"

From the Wall Street Journal via MSN, November 24:

Sam Altman wants artificial intelligence to cure cancer. Elon Musk says AI robots will eliminate poverty.

China is focused on something more prosaic: making better washing machines.

While China’s long-term AI goals are no less ambitious than the U.S. tech titans, its near-term priority is to shore up its role as the world’s factory floor for decades to come. With exports under threat from rising costs at home and tariffs abroad, that is no longer assured.

The push can be seen across the giant country in scores of companies—fueled by billions of dollars in government and private technology development—that are transforming every step of making and exporting goods.

A clothing designer reports slashing the time it takes to make a sample by more than 70% with AI. Washing machines in China’s hinterland are being churned out under the command of an AI “factory brain.”

At one of China’s biggest ports, shipping containers whiz about on self-driving trucks with virtually no workers in sight, while the port’s scheduling is run by AI.

Executives involved in China’s efforts liken the future of factories to living organisms that can increasingly think and act for themselves, moving beyond the preprogrammed tasks at traditionally-automated factories. It could further enable the spread of “dark factories,” with operations so automated that work happens around the clock with the lights dimmed.

The advances can’t come quickly enough for Communist Party leaders, who fear China could lose its status as the world’s factory floor. Its population is shrinking, young people are avoiding factory jobs, and pushback against Chinese exports has intensified in many countries. 

At the same time, President Trump is pledging to bring home vast numbers of manufacturing jobs through tariffs on China.

AI offers a lifeline to head off those risks, by helping China make and ship more stuff faster, cheaper and with fewer workers. Although some doubts are creeping in globally about how quickly AI will transform the world, China isn’t waiting: It wants to deploy what is available today quicker than the U.S. can, locking in any advantages.

“Only by proactively embracing change can we remain invincible in this revolution,” Hu Wangming, chairman of one of China’s largest steel groups, told state media this year. Its Shanghai-listed unit Baosteel had found 125 uses for AI by the end of last year—and is planning for 1,000.....

....Huawei’s role

Alongside DeepSeek, U.S.-sanctioned technology giant Huawei is at the center of China’s efforts. The company has rolled out a family of large language models, dubbed Pangu, after a character from Chinese mythology in the creation of the world, along with other AI services factories can use to become more dynamic.

Huawei’s engineers have been embedded with Conch Group, a giant cement producer in the city of Wuhu, around 200 miles west of Shanghai, whose cement has been used in projects such as the Three Gorges Dam and Dubai’s Burj Khalifa skyscraper.

Facing a glut of cement production in China, Conch wants a leg up on its rivals by adopting AI quickly, including in the production of clinker, a key cement ingredient made by heating limestone and other raw materials at high temperatures.

Conch and Huawei have developed AI tools for more precisely predicting clinker strength, as well as for controlling energy use at the kiln where it is made. Miles of conveyor belts in Wuhu are now being monitored by AI, helping Conch respond more efficiently when they break.

On a recent visit, workers at Conch’s sprawling clinker facility monitored the AI model as it automatically adjusted production in real time. Conch and Huawei say that with AI they are now able to predict clinker strength with more than 85% accuracy, compared with 70% through manual estimation, allowing them to adjust raw material ratios and avoid producing clinker of unsuitable strength....

....MUCH MORE 

What is happening with AI in China is exactly the path that nanotechnology followed. A vignette from a January 2019 post, "Whatever Happened to Nanotechnology? The Same Thing That Is Happening to Tech Right Now": 

a December 2010 post:
"Top Investment Trends For Futurists" (PXN; TINY)

...The reason for highlighting nano is two-fold.

1) Since Feynman coined the word there has been a misconception among investors that there would be a nano-technology "industry". This has proven not to be the case and won't be in the future. Rather nano is a tool, an approach toward problem solving.

There will be some breakthroughs that make their discoverers instantly (after 10 years of research) wealthy but the real beneficiaries will be companies like Kyocera and 3M and Siemens. They will use the technology to do what they are already doing, just better, faster, cheaper, more.

2) In spite of the fact that there will be few pure plays we are convinced that nano combined with advances in materials science and manufacturing technology is what will spur the next secular bull market....

When it becomes ubiquitous, the distinctions blur, the drive for creativity recedes, stasis, then death.
Wait what? Entropy! I meant to co-opt the physically precise  concept of entropy to metaphorically describe the trend. Not death.
No, death bad, Sand Hill Road good....

And AI?

I sometimes forget that normal people haven't been obsessing about this stuff for going on a decade. As just one example, out of hundreds of posts on NVDA, is a conclusion we reached early on, still hold, and tried to express in the introduction to a 2020 IBD story:

Investor's Business Daily on Artificial Intelligence Stocks

There is a definitional problem with the term "AI stocks [or companies]" in that AI is a tool. Much as the (over) hyped nanotechnology revolution didn't produce "nanotech stocks" but instead became incorporated into processes and procedures that give companies employing same an incremental edge rather than epochal shifts.*

However, if there is an AI "company" Nvidia would deserve the moniker as much as anyone....

Last seen in September 2024's "How Nvidia Is Building A Competitive Moat To Fend Off AI Challengers" (NVDA).

And Huawei? They are the people keeping Nvidia's Jensen Huang up at night, not the TPU folks at Google. If interested see:

"Huawei Plans Three-Year Campaign to Overtake Nvidia in AI Chips"

Chips: "Huawei lays out multi-year AI accelerator roadmap and claims it makes Earth’s mightiest clusters"

This is the one Nvidia's Jensen Huang thinks about 

As noted earlier this year:

What Huawei has accomplished is astounding and borderline terrifying. 

During the mid-to-late twenty-teens the world began to notice that the company was a serious business competitor and an extension of China's Communist Party and government.

In an attempt to assuage these concerns one of their Western honchos said:

‘At Huawei, we’re not attaching laser beams to the heads of sharks’
—Alykhan Velshi, Vice President, Corporate Affairs, Huawei Technologies Canada, Markham, Ont.
Letter to the Editor, Maclean's Magazine, published July 23, 2019

That was in one of our posts on China's security laws which require Chinese (and foreign-domiciled!!) companies to work with the authorities when asked. I'm not sure the statement did much assuaging....

From The Register, September 18: 

On the same day that fellow Chinese giant Tencent says its overseas cloud clientele doubled 

Chinese tech giant Huawei has kicked off its annual “Connect” conference by laying out a plan to deliver increasingly powerful AI processors that look to have enough power that Middle Kingdom users won’t need to try getting Nvidia parts across the border.

Huawei already offers the Ascend 910C accelerator that Chinese AI upstart DeepSeek is thought to have used to develop its impressively efficient models. At Connect today, Huawei promised four successors.

First off the rank, in the first quarter of 2026, will be the Ascend 950PR which, according to slideware shown at the conference, will boast one petaflop performance with the 8-bit floating-point (FP8) computation units used for many AI inferencing workloads. The chip will also include 2 TB/s interconnect bandwidth and 128GB of 1.6 TB/s memory. In 2026’s final quarter Huawei plans to deliver the 950DT, which will be capable of two petaflops of FP4 performance thanks to the inclusion of 144GB of 4 TB/s memory.

In 2027, Huawei plans the Ascend 960 that will include 288GB of 9.6TB/s memory. 2028 will see the debut of the Ascend 970, in which memory will speed along at 14.4 TB/s.

Those memory speeds suggest Huawei has created its own high-bandwidth memory, or sourced some from within China, and is confident enough to include it on a multi-year roadmap....

....MUCH MORE 

If interested the introduction was part of the outro from "The Secret History of China’s Most Powerful Company".

Some of our previous posts on the Chinese colossus:

While some people are bleating and tweeting into the ether about their political feelings, others are creating the technical container that will define, delineate, create, and constrict the future. Them's the ones to watch out for.
Baaaa
At the moment I don't think ASML has to worry but U.S. policymakers should be dusting off their contingency plans. They have contingency plans, right? I think they're in the same drawer as the CDC's pandemic response folder....

And many, many more. Quite an amazing story. 

And not afraid to get their hands dirty when the survival of the company was a talking point in 2021: Huawei Seeks Other Revenue Streams Including Coal Mining and Pig Farming

Also: