Am overview but an overview with insight.
From EE Times, December 9:
Nvidia’s ongoing fight to maintain technological and market dominance in AI inference.
For the past ten years, Nvidia has dominated the market for advanced computer chips used in machine learning and artificial intelligence.
Driven by its proprietary CUDA software and rapid innovation, Nvidia became synonymous with AI processors, briefly reaching a $5 trillion market value this year. Between February and October 2025, Nvidia reported $147.8 billion in sales from chips, network connections, and related hardware supporting AI growth.
Even though Nvidia’s newest and most powerful processors, the Grace Blackwell series, are selling out quickly, its dominance is starting to weaken. Instead of facing just one competitor, Nvidia is now challenged on many fronts as the industry shifts toward more specialized hardware.
Nvidia’s strong sales and high profit margins are driven by scarcity, primarily due to manufacturing limits. The main bottleneck for high-end chips is the limited capacity for advanced Chip-on-Wafer-on-Substrate (CoWoS) packaging at TSMC.
Nvidia consumes most of this limited production, but competition for chip supply is intensifying. TSMC, the sole manufacturer, plans to expand capacity to 100,000 wafers per month by 2026. As supply constraints ease, companies like Google and AMD are expected to benefit.
Nvidia now faces significant risk as the industry shifts from experimenting with large foundation models to prioritizing large-scale, cost-effective inference.
Major cloud providers are moving away from reliance on Nvidia’s CUDA ecosystem. They are investing in their own specialized chips for high-volume inference, where operating costs now exceed training costs.
Nvidia’s largest customers become competitors
The “Big Four” North American hyperscalers—Google, Amazon Web Services, Microsoft, and Meta—account for the largest capital expenditures in this sector. Their collective shift to custom silicon is a strategic move to ensure competitiveness.
Google, part of Alphabet, initiated the move toward custom AI chips with its Tensor Processing Units (TPUs). The latest version, TPU v7 ‘Ironwood,’ is optimized for inference. Ironwood features a large shared memory, enabling up to 9,216 chips to connect in a single ‘superpod,’ which addresses memory constraints in large Mixture-of-Experts (MoE) models.
Meta Platforms may lease and potentially purchase Google’s TPU chips for its data centers starting in 2027. This would mark a shift for Google, positioning it as a merchant chip supplier. Some estimates suggest Google could capture up to 10% of Nvidia’s annual revenue, amounting to billions of dollars.
Amazon focuses on training and cost efficiency
Amazon Web Services (AWS) is pursuing improved price-performance to attract businesses seeking alternatives to Nvidia’s expensive chips. AWS claims its Trainium chips can reduce training costs by up to 50% compared to GPUs, targeting mid-range AI workloads....
....MUCH MORE
