Samsung Electronics has boosted the speed at which artificial intelligence generates responses by more than three times using processing-in-memory (PIM) technology, which performs computing functions inside the memory chip itself. Analysts say competition in AI chips is expanding beyond raising the computing power of graphics processing units (GPUs) toward cutting the time data spends in transit.
Samsung disclosed the real-world AI inference performance of its LPDDR5X-based PIM at Hot Chips 2026, held at Stanford University in the United States on the 25th, according to industry sources on the 26th.
PIM embeds computing functions within memory. Memory, which had been limited to storing data, handles some calculations directly, reducing the burden on the processor.
In conventional AI chip architectures, a GPU repeatedly retrieves data stored in memory, performs calculations and sends the results back to memory. PIM, by contrast, processes some of the required calculations inside the memory, shortening the distance data must travel. That raises processing speed and lowers power consumption.
Samsung said its low-power double data rate 5X (LPDDR5X)-PIM places one PIM block, a cluster of computing circuits, for every 16 DRAM banks, for a total of 16 computing blocks. A bank is an independent storage space inside DRAM that holds a portion of the data, and the design allows each bank to perform calculations simultaneously.
The core of the announcement is that Samsung verified the AI performance gains on an actual manufactured chip. The company connected both conventional LPDDR5X and LPDDR5X-PIM to its own edge AI accelerator system-on-chip (SoC) and ran Meta's large language model Llama 3.1 8B.
LPDDR5X-PIM generated 81.3 tokens per second, 3.01 times faster than the 27 tokens per second of conventional LPDDR5X. The time required for the same task fell to 5.4 seconds from 12.3 seconds.
Tokens are units of data, such as characters or words, that AI uses to generate responses. That means inference speed can be raised significantly by improving the memory architecture, even when using the same AI model.
Samsung had previously disclosed an LPDDR5X-PIM prototype and its basic performance. This announcement is significant in that the company confirmed performance gains by running a commercial AI model on an actual chip and disclosed the internal computing architecture in detail.
The design also takes commercialization into account. Samsung used an address alignment mode (AAM) to allow LPDDR5X-PIM to work with existing DRAM memory controllers. A memory controller manages data input and output between the processor and memory. That lowers the burden of having to completely overhaul existing system architecture to adopt PIM.
Samsung also said the first formal specification for its next-generation LPDDR6-PIM at the Joint Electron Device Engineering Council (JEDEC) is nearing finalization. The company plans to expand the technology beyond a proprietary offering into a standard memory technology that a range of system makers can use.
An industry official said the LPDDR5X-PIM that Samsung unveiled at Hot Chips, a leading global semiconductor conference, represents a structural change in which some AI computation is handled directly where the memory sits. The official added that competition in AI chips is moving into a stage where how efficiently data is moved and processed matters, not just how fast a GPU can compute.
