What Samsung Electronics and SK hynix ultimately aim for through Compute Express Link (CXL) memory development is the expansion of a new "memory tier." Currently, high-bandwidth memory (HBM) and double data rate (DDR) memory, which store artificial intelligence (AI) data, face capacity limits. HBM and DDR are not only in short supply, but even when inventory exists, they are difficult to add freely due to the AI chip architecture in which they are mounted directly onto graphics processing units (GPUs) and central processing units (CPUs), respectively. Because the number that can be physically connected to GPUs and CPUs is limited, increasing HBM and DDR requires first increasing the GPUs and CPUs themselves. In that case, cost burdens rise sharply.
By contrast, CXL memory forms a memory device separate from HBM and DDR, that is, a new memory tier, expanding capacity at relatively low cost. It has the drawback that data communication speed declines when it is not directly connected to GPUs and CPUs as HBM and DDR are, but CXL, a new communication technology called the "data highway," compensates for this. Even when DRAM is added without being directly connected to GPUs and CPUs, it can support computation at relatively smooth speeds with latency of several hundred nanoseconds (a nanosecond being one billionth of a second). Accordingly, the two companies are conducting all-around development of not only new products but also architectures (design structures) that can organically combine these products with existing memory tiers to create synergy.
According to the industry, SK hynix (000660.KS) developed and unveiled to academia last month an "Inference Memory Tier Expansion (IMTE)" architecture technology. The company explained that it succeeded in improving AI inference efficiency by 35.7% compared with before by adding "CXL hybrid memory" between HBM/DDR and solid-state drives (SSDs) and designing them to create synergy with each other.
Current AI memory systems have a tiered structure of "HBM→DDR→SSD." HBM and DDR store the "hot data" that GPUs and CPUs, respectively, need for immediate computation. The remaining large volumes of "cold data" not immediately needed are stored long-term by SSDs. HBM has a small capacity of several tens to hundreds of gigabytes (GB) per unit but operates at fast speeds. DDR reaches several terabytes (TB) and SSDs several tens of TB, growing in capacity but slowing in speed.
CXL memory, with capacity of several to several tens of TB and relatively fast speed, shows characteristics intermediate between DDR and SSDs. SK hynix's architecture thus expands the memory tier structure into "HBM→DDR→CXL memory→SSD." In particular, this architecture uses CXL memory beyond simple capacity expansion, as a hub that organically connects HBM/DDR and SSDs. CXL memory predicts in advance the hot data needed for HBM and DDR, and retrieves that data from the SSD ahead of time to deliver it to HBM and DDR, helping the SSD serve as a computation aid beyond being merely a long-term storage device.
Samsung Electronics (005930.KS) also presented optimization technology through a CXL technology white paper on the 9th of this month. Samsung Electronics introduced that its CXL 2.0 memory (CMM-D 2.0) can achieve AI inference performance approaching 92% of the level of DDR5, the latest DDR product. While CXL memory is more advantageous than DDR for capacity expansion, its operating speed inevitably falls short, but the company explained that it overcame this weakness through optimization technology. Samsung Electronics is also developing "CMM-Hybrid (H)," which applies CXL not only to DRAM but also to NAND, similar to SK hynix's CXL hybrid memory. Micron, too, is known to be currently accelerating internal development of new products based on CXL 3.1 or 3.2 R, after unveiling its CXL 2.0 memory product "CZ120" in 2023.
As development competition among global memory companies intensifies to seize leadership, market research firm Yole Group forecast that the CXL market size will grow more than 652%, from $2.1 billion (about 3 trillion won) this year to $15.8 billion (about 23.5 trillion won) in 2028.
As AI data explodes, the move to expand memory tiers such as CXL is also accelerating. Global investment bank Goldman Sachs projected that worldwide monthly AI token usage will surge 24-fold, from 5 quadrillion this year to 120 quadrillion in 2030. Each token typically occupies four letters of the alphabet, that is, 4 bytes (B) of capacity. The capacity that AI chips require to run the latest large language models (LLMs) has already swelled to the level of several tens of TB. SK Group Chairman Tae-won Choi also explained at a recent press conference marking SK hynix's Nasdaq listing that "the more AI is used, the more token usage increases enormously, and this means that the KV cache, which stores computation results, keeps growing."
An industry official said, "CXL memory is still in the early stages of commercialization, but like HBM, customers can need it at any time, so suppliers are proactively developing technology and preparing for competition." Major big tech firms such as Nvidia, Intel and AMD are also responding to the paradigm shift by supporting CXL memory compatibility in new chips such as "Vera Rubin," to be released in the second half. Microsoft (MS) also piloted the launch of the first cloud service equipped with CXL memory to boost server efficiency late last year.
In a similar vein to CXL, high-bandwidth flash (HBF) is also being actively developed as a candidate for a new memory tier. Just as HBM stacks DRAM vertically, HBF is a technology that stacks NAND vertically to simultaneously secure large-capacity memory and high bandwidth. In particular, SK hynix, anticipating that demand will grow in earnest around 2030, is pursuing technology standardization in partnership with SanDisk.
Nvidia plans to introduce for the first time a new NAND-based memory tier called "Compute Memory Expansion (CMX)" in its next lineup, "Vera Rubin." AI chips recently unveiled by big tech firms, such as Groq's language processing unit (LPU) and Google's tensor processing unit (TPU), are also showing moves to significantly expand SRAM capacity, which is faster than DRAM, and to secure it as a new memory tier.
