CMM-H (CXL Memory Module – Hybrid): Rethinking Storage for the Memory-Centric Computing Era

In May of 2021, Samsung announced the development of the industry’s first CXL DRAM, known as CMM-D (CXL Memory Module DRAM). Designed to support CXL™ 2.0, this first offering touted 128GB CXL DRAM, leveraged the PCIe 5.0 interface and achieved a remarkable bandwidth of up to 35GB per second. In May of 2022, Samsung introduced the next generation, CMM-D 2.0, featuring the industry’s first 512GB Memory Module, along with high bandwidth and reduced latency. This was a pivotal step in the commercialization and practical application of CXL technology. The vast potential of AI, particularly with the growth of large language models like GPT, necessitates a significant expansion in memory capacity. However, simply adding more memory is not a sufficient solution because memory performance does not scale linearly with capacity. In addition, current memory modules struggle to keep pace with rapidly evolving processor and accelerator speeds. The memory limitation often results in frequent data migrations, which consumes substantial power and increases total cost of ownership (TCO). The root of this issue lies in the “traditional” processor-centric configuration of modern data centers. In other words, there is no universal processor-centric solution that solves low system performance, high TCO, and challenges such as bandwidth, latency, capacity, and high-power consumption. The tiered memory model offers an architectural solution to this complex problem. By strategically positioning frequently accessed data closer to the processing units, it not only effectively expands memory capacity but also enhances cost efficiency. In other words, placing memory where the data is stored will enable faster data processing, lower power requirements and reduced TCO. CMM-H (CXL Memory Module – Hybrid) Samsung Semiconductor is leading innovation in memory technology, particularly addressing processor-centric limitations in AI/ML workloads. Their latest venture, the CMM-H (CXL Memory Module – Hybrid), emerges as a next-generation memory solution optimized for tiered memory systems. The CMM-H features Samsung’s high-performance DRAM, coupled with NAND Flash, and a CXL interface that built upon a PCIe Gen 5.0 interface, offering a cost-effective memory expansion.

A key element of CMM-H is its built-in DRAM cache, designed to mitigate the long latency associated with NAND flash. In modern data processing systems, each processer has its own cache that stores frequently accessed data to increase a speed. Similarly, this built-in DRAM cache, efficiently caching of frequently accessed data offers performance comparable to host DRAM with 100% cache hit ratio. CMM-H further sets itself apart with its 64-byte cache granularity access, leveraging the revolutionary CXL.mem protocol. This protocol is a game-changer for AI applications. Unlike a traditional NVMe device, which requires data swapping, moving data back and forth from the SSD to host DRAM, CMM-H facilitates direct data access enhancing the data processing efficiency, lowering latency by providing a more efficient I/O stack and increasing effective bandwidth through smaller granularity for data access. This was possible by cache coherency enable by CXL technology (CXL.mem). Cache coherency used to be available only among main memories or main memory caches for processors, now available for CXL devices so that all processors sharing access to the common memory see the same up to date version of the data. Moreover, the versatility of CMM-H is noteworthy. It operates not only as a memory expansion device to meet the demands of tiered memory systems but can also serve as a PMem device for persistent memory needs. The persistence feature is enabled through data dumps to NAND flash, and the CMM-H supports the flush-on-fail capability with the CXL 2.0 GPF. Dr. Rekha Pitchumani explains in great detail of CMM-H processes with illustrations and demonstrations in her presentation, “CMM-H: Samsung’s CXL-based SSD for the Memory-Centric Computing Era.” _{Note: Memory-Semantic SSD has been renamed to CXL Memory Module – Hybrid, or simply CMM-H.} More information regarding CXL can be found in the Tech Blog, “Scaling Memory Capacity and Bandwidth in AI/ML and HPC Applications with CXL Technology”.