Big Tech Is Gunning for Memory

A new narrative structure has recently emerged in the memory industry: the "contract-based industry." Fundamentally, cyclical industries that produce goods and then sell them are vulnerable to economic conditions and inventory. DRAM prices, for example, ride a roller coaster depending on supply and demand. As a result, operating margins that soared to 40 to 50 percent during boom periods routinely turned into losses during downturns.

But a cheat key called artificial intelligence (AI) has appeared, and everything is changing. High-bandwidth memory (HBM) and some high-performance memory, which have already risen to the center of AI computing, are increasingly locked into long-term supply contracts. If processors were once the protagonists and memory the extras, now the balance of power is shifting.

In fact, not long ago, Micron CEO Sanjay Mehrotra publicly took aim at Apple, saying that memory companies could not invest in a timely manner because of Apple's bullying in beating down memory prices, and that the current memory shortage is the result. Samsung Electronics and SK hynix probably felt a sense of catharsis at Mehrotra's pointed remarks. Apple took the humiliation as the representative, but within "Apple" one must count all the giant memory buyers such as Nvidia, Intel, AMD, Google, and Amazon.

The problem is that the stronger memory becomes, the more organized and blatant Big Tech's moves to reclaim power may become. Google sent chills down memory companies' spines with TurboQuant, a software algorithm that reduces memory usage in the AI inference process — and that is merely a taste of what is to come. Qualcomm has introduced HBC to lower HBM usage, and Intel has introduced a technology called ZAM to reduce data movement, one after another.

That is not all. Corporate moves to reduce memory usage through acquisitions of software optimization companies, as AMD has done, are also clear. There is a need to keenly recognize that, faced with an era of memory scarcity, Big Tech firms are actively involving themselves in architecture design to create memory structures in which AI operates most efficiently. It means that the scarcer memory becomes, the more a war on an entirely different dimension is breaking out on the other side to build AI that uses less memory. Some forecast that these various technologies to reduce the memory bottleneck will conversely lower memory cost burdens and further increase demand for HBM and the like. But what is clear is that Big Tech is grinding its teeth over algorithm and system innovation. The goal is an intention no longer to tolerate the risk of being subordinate to memory vendors.

In fact, long-term memory supply contracts are not permanently possible either. Memory has now reached the status of a strategic asset that determines the utilization rate of AI chips such as graphics processing units (GPUs). The decisive cause of the most expensive GPU in an AI server sitting idle is a shortage of HBM, and for this reason Big Tech firms are securing memory through long-term supply contracts even at high cost. This is because the loss is greater if GPU utilization falls while trying to save on a relatively small share of memory costs. Considered carefully, the current memory price is closer to the cost of preventing GPU idle time. That is why it can be more expensive than actual supply and demand would suggest.

But the day to verify AI monetization is bound to come. Big Tech firms, now bent on securing memory, will at some point ask the essential question of return on investment. Setting aside macroeconomic changes such as interest rates, it is also hard to rule out the rapid growth of CXMT, which has begun supplying server DRAM to Chinese Big Tech, and the possibility of spite from a U.S. government uncomfortable with Korea's memory supremacy.

One must look beyond the boom. It must not stop at "sell more HBM." The board must be set so that our technology is absolutely necessary even when less HBM is used. To that end, there is a need to proactively expand influence over memory architecture itself, including CXL, a technology that shares memory like a network. If we remain a mere memory supply company, drunk on a windfall boom, we could hand leadership of the AI memory ecosystem back to Big Tech.

The most dangerous moment in tech history has been when one stood at the center. The moment we cheer that "the memory era has come," someone is preparing the next technology to bring down that center. What our companies must truly guard against is not today's supply shortage but tomorrow's technological change. Only companies that can stand at the center of that change, whatever direction memory evolves in, can become the final winners of the AI era.