The entire AI boom rests on a very specific kind of chip. It's the most-searched layer and the least understood. Here we explain, without jargon, what makes AI chips special, what the famous HBM memory is, and why NVIDIA's dominance is so hard to break.
GPU versus CPU: why the switch
A computer's normal processor (the CPU) is like a genius solving one complex problem at a time, very fast. A GPU is the opposite: thousands of simple workers doing calculations at once. Training an AI is repeating millions of similar operations, so the GPU's "parallel" work fits perfectly. That's why NVIDIA, which had spent decades making GPUs for video games, found its chips were exactly what AI needed.
The hidden bottleneck: HBM memory
A powerful chip is useless if it can't be fed data fast enough. That's where HBM (high-bandwidth memory) comes in: ultra-fast memory stacked next to the chip. It's expensive, scarce and made by very few companies, which makes it a key business within the boom. In the U.S., Micron is the big listed memory name (the other giants, Korea's Samsung and SK Hynix, aren't on U.S. exchanges).
NVIDIA's real moat: the software
Here's the key many people miss. NVIDIA's dominance isn't explained by having the best chips alone, but by CUDA: the software platform developers have used to program its GPUs for more than fifteen years. A whole generation of AI engineers learned on CUDA. Switching to another maker forces a lot of rewriting and relearning, so the moat isn't the silicon: it's the ecosystem. It's the same thing that makes it so hard to leave other platforms you're "locked into".
The challengers
- AMD: the most direct rival in data-center GPUs, trying to offer a credible alternative to NVIDIA.
- Broadcom: instead of competing head-on, it designs custom AI chips for giants like Google or Meta that want to reduce their reliance on NVIDIA. It also dominates the networking between chips.
- Big tech's in-house chips: Google (TPU), Amazon and Microsoft design their own chips for their clouds. They're not bought separately, but they pressure NVIDIA from below.
And who makes all of this
Neither NVIDIA nor AMD owns factories: they design the chips, and Taiwan's TSMC (not U.S.-listed) manufactures them. And TSMC buys its machines from a handful of equipment suppliers, among them Applied Materials and Lam Research: the "picks and shovels" that win no matter whose chips sell.
How to invest in this layer wisely
Chips are the heart of AI, but also the most expensive corner of the market right now. Before buying, check whether the price keeps up: type the ticker into the analyzer to see its P/E against its growth. And to place this layer within the whole, go back to the map of the companies behind the AI boom.