Pioneering The Next Memory Frontier

High-performance computing (HPC) and AI are transforming the semiconductor landscape beyond recognition. The demand for memory density and bandwidth is increasing faster than ever, to a point where we need to rethink our approach to memory in order to break free of the memory wall.

Innovation is key, at Samsung, we’re primed to respond to these shifting computational paradigms thanks to our pioneering work in high-performance memory.

 

Breaking the AI memory wall

Data processing requirements in the age of machine learning and AI model inferencing are ever more complex, demanding extra capacity and tighter precision. The bar for what memory can do keeps rising, creating what we call a “memory wall”.

So what is the memory wall and why do we need to break it?

The memory wall gets built when insufficient memory in AI applications results in delayed data transfers, causing the system to work harder and consume more power, ultimately increasing the total cost of ownership.

Samsung’s “Near Memory Solutions” are designed to tear down this wall through a revolutionary approach: in-memory processing. In-memory processing is where memory actually handles some of the data processing workload. This evolution allows memory solutions to shoulder some of the burdens traditionally handled by CPUs and GPUs – marking a new frontier for managing and leveraging vast amounts of data.

In addition, PIM is our next-generation convergence technology that improves energy efficiency for AI accelerator systems by reducing data transfer between the CPU and memory. HBM-PIM has doubled performance and reduced energy consumption by around 50% on average, in comparison to the previous generation.

We’ve been collaborating with major data center customers and CPU/GPU leaders to develop the industry’s first Compute Express Link (CXL™) based DRAM Memory Module (CMM-D). While adding memory capacity and bandwidth in a system usually involves increasing the number of native CPU memory channels – thereby increasing engineering complexity and cost – a CXL Type 3 memory expansion device ramps up memory capacity and bandwidth without the need to do this.

 

The future is now

The high-performance memory market is poised for rapid transformation. Therefore, we are proactively creating and launching products ahead of demand to address current and future memory needs. Simultaneously, we’re also focused on ensuring the ongoing reliability and sustainability of our products – testing technologies and making efficiency improvements that will power increasingly sophisticated seamless AI services. 

“As the world moves from using “Structured Data” to an AI-driven world of looking at all Unstructured Data, the processing requirements are increasing dramatically. Memory needs to move from the evolutionary approach of the past years to a more dramatic change using concepts like PIM to remove the processing bottleneck, while increasing bandwidth and lowering power consumption. The ultimate goal of doing more for less and with this, reducing TCO in the process.” Richard Walsh – Head of Memory Marketing at Samsung Semiconductor Europe

It's an incredibly exciting time for memory and its vital role in shaping our technological future. AI has already had a significant impact on our lives in diverse ways – and who knows what changes it will bring in the next few years. Continuing to push the envelope in memory innovation will help us to create a bigger, bolder future. A future that we’ll never forget.