Nvidia (NASDAQ: NVDA) has been around since 1993, but it came into the spotlight for its graphics processing units (GPUs), which are crucial for training and scaling artificial intelligence (AI). They provide much of the compute power that powers AI workloads.
At the beginning of the current AI boom, the goal for tech giants was simply acquiring as much compute power (i.e., GPUs) as possible. Now, the focus has shifted to memory chips, but as Nvidia’s CEO, Jensen Huang, highlighted, those memory chips are now AI’s biggest bottleneck.
Missed Nvidia in 2009? This Rare Signal Is Flashing Again. In 2009, a “Double Down” signal flashed for a little-known chipmaker called Nvidia. For the first time in years, that same “Total Conviction” signal is flashing for a company 1/100th the size of Nvidia. Continue »
Why memory is important to AI and Nvidia
AI training and application rely on trillions of data points, and it wouldn’t be possible to store and quickly retrieve them without specialized memory chips. As AI is used for handling more complex tasks — such as running autonomous agents or processing complicated context instead of providing recipes or travel recommendations — the need for high bandwidth memory has become increasingly important.
Nvidia began its reign selling its GPUs and AI hardware, but now it’s building systems with multiple working parts, including memory chips that are packed into its hardware. That means relying on memory chipmakers, such as Micron, SK Hynix, and Samsung, for a continuous, high-volume supply.
Unfortunately, making those specialized memory chips is far from simple, which is why only a handful of companies make the vast bulk of them.
What does it mean for Nvidia?
The downside to the shortage is that Nvidia is at the mercy of memory chip suppliers for its own supply chain. If the suppliers can’t make memory chips fast enough, Nvidia will have to wait, potentially affecting its own business.
The positive is that Nvidia has the cash and purchasing scale to have priority on the memory chips being made. In its most recent quarter (ended April 26), it generated $48.6 billion in free cash flow and finished the quarter with $13.2 billion in cash and cash equivalents. It can easily pay a premium to buy them in bulk, shutting out smaller competitors and further cementing its stronghold on the industry.
Investors shouldn’t hear Huang’s message and become concerned; it’s just the next chapter of the AI evolution. If anything, it should be encouraging that Nvidia can use its leadership position to be a long-term force, regardless of the current hiccup.
