Marvell Unveils AI Memory Strategy to Boost Server Efficiency. What That Means for MRVL Stock.

Ai chip by Quality Stock Arts via Shutterstock Marvell Technology (MRVL) has spent the past decade building chips that move, store, and process data for the world’s biggest cloud companies. Now, the chipmaker is turning its attention to a problem that is slowing down artificial intelligence platforms. As AI models grow bigger and chatbots hold…


Marvell Unveils AI Memory Strategy to Boost Server Efficiency. What That Means for MRVL Stock.
Ai chip by Quality Stock Arts via Shutterstock
Ai chip by Quality Stock Arts via Shutterstock

Marvell Technology (MRVL) has spent the past decade building chips that move, store, and process data for the world’s biggest cloud companies. Now, the chipmaker is turning its attention to a problem that is slowing down artificial intelligence platforms.

As AI models grow bigger and chatbots hold longer conversations, the computers running them are running out of fast, accessible memory. Marvell says it has a plan to address this issue while accessing every layer of a data center, from a single server to an entire building.

More News from Barchart

www.barchart.com

Why AI Models Are Hitting a Memory Wall

Modern AI chatbots answer questions and remember every conversation. Every word a user types gets stored temporarily in something called a KV cache, short for key-value cache, so the AI can refer back to it.

The problem is that these caches are getting massive. Longer conversations, bigger AI models, and the rise of AI agents that complete multistep tasks on their own require more memory than a single server chip can hold.ย 

When a processor has to wait for data because memory is full or too far away, it sits idle. Marvell calls this a GPU stall, and it’s expensive. Marvell announced new products aimed at this bottleneck, and the lineup spans three levels of a data center.

  • At the individual server level, the company’s new Bravera SC6 chip controls solid-state drives and helps move overflow data from memory out to storage twice as fast as its predecessor, according to the statement.

  • At the rack level, a product called Structera X lets cloud providers pool memory across multiple servers instead of locking it inside each machine. That means less memory sits unused while another server nearby runs short.

  • At the broadest level, Marvell’s Photonic Fabric technology uses light instead of electrical signals to link memory across racks up to 50 meters apart.ย 

This setup can offload up to 32 terabytes of cached data and deliver two to three times more AI output within the same power budget.

“AI infrastructure is moving beyond isolated servers to systems where compute, memory and connectivity operate seamlessly together,” said Will Chu, executive vice president and general manager of custom cloud solutions at Marvell, in the statement. “As AI scales, memory must scale more independently of compute so resources can be deployed where they deliver the greatest value.”

Source link