Han Sang-wook, a technical leader (TL) on the DRAM design team of Samsung Electronics' Memory Business Division, presented on how the HBM base die can boost AI computing capability at the semiconductor seminar "Hot Chips 2026," held at Stanford University in California on the 23rd (local time). The base die is a logic chip at the very bottom of HBM, where DRAM is stacked, that controls speed and power.
Han cited the limits of through-silicon vias (TSV) and physical layer (PHY) input/output as the main bottlenecks blocking the expansion of memory bandwidth (data transferred per second). TSV is a packaging technology that grinds DRAM chips down to less than half the thickness of ordinary paper, then, rather than connecting the chips with wires, drills fine holes to link the top and bottom of the chips with electrodes. PHY handles data movement between the AI chip and memory. The problem is that TSVs and PHYs cannot be increased indefinitely within a small memory.
Samsung aims to solve this with custom HBM (cHBM). The design reinforces only the necessary functions, such as speed or power efficiency, by tailoring the base die to the graphics processing units (GPU) and neural processing units (NPU) used by customers such as Nvidia and AMD. He added a plan to change the PHY to a die-to-die connection structure to make room for these custom functions. Han also introduced a concept of transplanting memory control and some AI computation now inside the GPU into the base die. "It started from the question of why, as the difference between the base die and the system-on-chip (SoC) shrinks, we don't make the base die functions like an SoC," Han explained.
SK hynix presented packaging technology as its solution. Lee Jae-sik, senior vice president of SK hynix America, stressed, "Going forward, we must optimize not only HBM performance itself but also the GPU, packaging, and foundry together with customers."
HBM is mounted on a single substrate along with other semiconductors such as GPUs and then goes through packaging; since high heat and physical stress arise in this process, the HBM manufacturer must ensure that the HBM stably maintains its performance and structure even after passing through all the processes.
"The solution to the heat problem is to change the structure and materials," Lee said. "There are many challenges to overcome to stack memory 20 layers high. We're not sure whether it will be applied to HBM4E (7th generation), but we are researching the hybrid bonding technique."
Hybrid bonding is a process that directly connects metal pads between chips without an intermediary material, allowing memory to be stacked while minimizing heat generation and increasing bonding strength. Lee explained that hybrid bonding can narrow the TSV circuits more than existing techniques and expand the thickness of the core die, which makes up most of the HBM, by 24%. He also shared the development status of "iHBM" technology, which adds a high-thermal-conductivity material to cool heat that arises only in specific areas inside the HBM.
