Skip to content

Empowering AI: Samsung Showcases Next-Gen Memory Solutions at the 2024 OCP Global Summit

  • mail

As the AI revolution transforms the data center and server memory landscape, the demand for high-capacity and high-bandwidth solutions grows exponentially. This trend challenges conventional memory technologies and architectures, pushing the industry towards innovative solutions.

 

At the 2024 OCP Global Summit, held October 15-17 in San Jose, California, Samsung Electronics highlighted its next-generation memory technologies, demonstrating how these solutions can be implemented into partner collaborations to bring about real-world impact while addressing AI’s growing demands.

 

As the desire for data-intensive workloads continues to rise, Samsung Electronics is once again at the forefront, unveiling cutting-edge solutions across memory and storage technology. At the core of their latest advancements are Compute Express Link (CXL) solutions, complemented by breakthrough developments in DRAM and Storage Solutions.

Image 01
Image 01

Samsung’s Next-Gen CXL Solutions

To meet the rapidly increasing demands of the AI industry, Samsung unveiled a range of Compute Express Link (CXL) solutions, including CMM-D, CMM-H, and CMM-B, which address the limitations of conventional memory architecture by allowing efficient resource management and memory expansion. These CXL solutions utilize CXL 2.0 technology to facilitate seamless communication between CPUs, GPUs, and accelerators, significantly enhancing performance in data center environments.

 

A key highlight at the show, was CMM-D (CXL Memory Module – DRAM), which is positioned to redefine server memory scalability by offering substantial capacity and performance benefits. At the CMM-D Collaboration Corner, Samsung showcased partners’ server demos equipped with CMM-D to demonstrate the transformative impact of CXL memory in data-heavy environments.

 

By combining CMM-D with NAND Flash, Samsung offers CMM-H (CXL Memory Module – Hybrid) in a type of PM (Persistent Memory) and TM (Tiered Memory), optimizing memory capacity and bandwidth for next-generation applications. In order to enhance data transfer by addressing latency concerns, Samsung’s CMM-B (CXL Memory Module - Box) solution enables a smoother experience for complex workloads. Its ability to support extensive memory pools further ensures efficient handling of demanding data processes.

Leading the Memory Revolution: DRAM and Storage Innovations

While CXL takes center stage in Samsung's next-gen memory solutions, the advancements in DRAM and Storage technologies remain critical components of the company’s offering. HBM3E, for instance, is built to process the immense data demands of AI models, providing speeds of up to 1,250GB/s and boosting thermal efficiency by 12%. This makes it a key enabler for smooth and rapid data processing across AI applications.

 

Complementing this is RDIMM (Registered DIMM), which stabilizes high-performance systems by reducing electrical load, ensuring reliability under heavy workloads. For applications that demand even more bandwidth, MCRDIMM (Multi-Ranked Buffered Dual In-Line Memory Module) steps in, with data throughput up to 12.8Gbps (based on Gen2), providing a critical advantage for handling AI and high-performance computing tasks.

 

For AI acceleration and 3D rendering, the industry’s-first 24Gb GDDR7 is ideal. This memory solution utilizes 5th-generation 10-nanometer (nm)-class DRAM and increases cell density by 50% compared to its predecessor while maintaining the same package size. The Pulse-Amplitude Modulation (PAM3) signaling technique enables 25% faster data transmission, with speeds reaching 40Gbps, making it an optimal solution for next-generation applications like AI workstations, data centers, and autonomous driving.

 

In storage, Samsung’s PBSSD (Petabyte-class SSD) and PM9D3a SSD are engineered to meet the rising demand for cloud and enterprise solutions. The PBSSD integrates Samsung's latest V-NAND technology, offering enhanced scalability with configurations up to 928TB when combined with 26pcs of BM1743 and 6pcs of PM9D3a, for example. This solution features power optimization technology that reduces energy consumption by about 20%, increasing rack density in data centers. The PM9D3a SSD ensures high-speed performance and reliable data transfer as the industry’s fastest 8ch PCIe 5.0 SSD with OCP DC spec v2.5 in mass production, while the BM1743 SSD demonstrates Samsung's commitment to scalable, high-performance storage solutions tailored for modern enterprise infrastructure.

Image 02
Image 02
Image 03
Image 03

Panel Discussion: Scaling Interconnect and Memory for AI Clusters

On October 15th,  Taeksang Song, Corporate Vice President and Head of New DRAM Solution Development Team at Samsung Electronics, took part in a keynote panel discussion focused on the challenges and solutions for scaling AI systems, in particular, AI interconnects and memory technologies. Other panelists included Nigel Alvares, Vice President of Global Marketing and Business Planning at Marvell, and Chris Petersen, Fellow of Technology and Ecosystems at Astera Labs. Together, they explored crucial topics surrounding memory hierarchies, the significance of CXL in scaling, and the essential role of reliable interconnects and cooling systems. Insights were shared on addressing latency issues in inference and the potential of CXL memory to tackle these challenges.

Image 04
Image 04

During the panel discussion, Taeksang shed light on the critical need for flexible and reliable AI solutions. He highlighted the significance of resource disaggregation and domain-specific observation in boosting performance. Moreover, Taeksang delved into the challenges posed by data center cooling systems, underlining the high energy consumption associated with cooling and the need to optimize these systems for enhanced energy efficiency.

Image 05
Image 05

In addition, Taeksang underscored the advancements in Samsung’s CXL solutions, particularly focusing on the CXL memory pool access latency and its efficacy in transferring large volumes of data seamlessly. He proposed that utilizing CXL memory in conjunction with processing near memory engines could potentially address the gap between memory capacity and latency, offering a promising solution in this realm. This innovative and scalable next-generation memory solution offers high capacity and high bandwidth, enabling seamless memory sharing and expansion without the need for additional infrastructure investments. This is particularly advantageous in the era of AI, where large-scale investments are often required.

 

 

Expanding Open Collaboration for the AI Era

To actively engage in CXL collaborations, Samsung established the OCP Experience Center at its Memory Research Center in 2022. This center provides opportunities for customers and partners to experience OCP-certified products and collaborate on diverse projects.

 

During the event, Samsung introduced its CMM-D reference architecture, showcasing real-world use cases that reduce total cost of ownership (TCO), developed in collaboration with Red Hat, Dell, and Broadcom, highlighting the memory expansion concept enabled by CMM-D.

Image 06
Image 06
Image 07
Image 07

Samsung is committed to driving innovation through industry-wide partnerships and actively contributing to the open-source ecosystem. By advancing hardware design and memory technology, it is fostering groundbreaking solutions that meet the evolving needs of AI and data infrastructure. As AI technologies continue to evolve, Samsung remains dedicated to collaboration across industry boundaries, bringing cutting-edge solutions to life.

Would you like to
leave this page?
If you leave this page, the content you are creating
will not be saved.

Registration Are you sure you want to submit this?

Thank you! Please confirm your registration

Your subscription is not active yet!
An email with an activation link
has just been sent to your email address.
Please activate your subscription by clicking on
the activation link inside the email.

Confirm
Thank you! Please confirm

your existing registration

You have already registered, but before we can send you the
information about upcoming events, we need your confirmation.

If you missed our previous email, please use the button below to resend it.
To activate your subscription, please click on the link included in the email.

Resend
Alert

To proceed, please click on the "check" button located in the email section.

Confirm