Redefining Possibilities in Memory Technology: Samsung CXL Memory Appliance with Orchestration Console

The dynamics of data consumption in the AI era are evolving rapidly, pushing current hardware and software capabilities beyond limits. Companies such as Tesla, Zoox, and Waymo are leading efforts to deliver autonomous driving solutions. Similarly, financial institutions like Mastercard and Visa are striving to implement real-time fraud detection. To unlock the financial potential of these innovations and provide value to consumers, businesses must analyze AI-generated datasets—both large and small—in real-time, without compromising on capacity, latency, or increasing TCO (Total Cost of Ownership).    

Samsung Semiconductor offers a solution aimed at helping companies achieve optimal TCO while maintaining high performance in terms of latency and capacity. Samsung’s software-defined memory technology solutions, powered by Compute Express Link (CXL), are designed to provide the scalability, manageability, configurability, and flexibility required as data centers transition into the next generation of disaggregated system architectures.

At OCP Global Summit and Supercomputing in 2024, Samsung Semiconductor showcased its CMM-B (CXL Memory Module – Box) and SCMC (Samsung Cognos Management Console). These innovations are tailored to meet the growing demands for efficient AI/ML-driven systems while ensuring optimal resource utilization, reduced memory access latency and minimized TCO. Together, CMM-B and SCMC provide a memory expansion, pooling and management solution, respectively.

1. Key Features of CMM-B and SCMC

1) CMM-B (CXL Memory Module – Box)
The CMM-B serves as a memory pool that supports CXL 1.1 and 2.0 protocols. It can host up to 24 x E3.S CMM-D devices, with total capacity ranging from 3TiB to 24 TiB, respectively. Additionally, up to 8 hosts can connect to a CMM-B. This capability allows dynamic and efficient memory allocation across multiple systems. Below is a detailed breakdown of its features. 

2) SCMC (Samsung Cognos Management Console)
SCMC is a user-friendly interface designed for managing CMM-B. It provides:

• REST API Support
• Visualization of Host Memory Usage & Statistics
• Topology View of CMM Devices, Switch, and Hosts

Graphics below demonstrate how Samsung Cognos Management Console works with CMM-B.

Above picture depicts SCMC integration with rackscale management systems. SCMC’s northbound API based on DMTF and Redfish specifications, allows convenient integration points with 3^rd party software. Above picture also depicts how SCMC agents are deployed on various server nodes in Samsung CMM-B racks which can be discovered explicitly over REST or through SSDP.

2. SCMC deployment and features

One of SCMC’s standout features is its ability to facilitate Memory Pooling, dynamically allocating memory resources, based on demand. Below graphics depict how SCMC simplifies this process while adhering to CXL specifications.

Above picture depicts a typical SCMC cluster deployment consisting of one or more management nodes and one or more data nodes. SCMC data node agents manage respective data nodes and provide solicited information on memory attached. Management nodes provide REST and graphical interfaces based on DMTF and Redfish standards for overall cluster and memory management. SCMC south bound infrastructure interacts with memory devices and is based on CXL specifications and interacts with one or more fabric managers. SCMC is capable of interacting with heterogenous devices through its flexible and pluggable architecture.

3. Benefits of CMM-B and SCMC

IMDG, AI/ML and other memory intensive applications can greatly benefit from increase in memory density per data node. This not only allows saturating CPU bandwidth but also network bandwidth by allowing larger data transfers.

In addition to memory expansion, it is also important to optimize TCO, efficiently utilize capacity with minimal compromise in data access latencies. Samsung CMM-B provides various memory expansion features, while SCMC compliments CMM-B by maintaining overall application’s data access latency within its tolerable end-to-end requirement. Some of the CMM-B + SCMC features that enhance the overall user experience can be illustrated as below,

• Disaggregated Memory Pooling and Sharing: Enables dynamic memory allocation for compute systems on demand.
• Dynamic and Static Configuration: CMM-B facilitates both static and dynamic configurations through the SCMC. SCMC manages CMM-B memory system including dynamic memory allocation/deallocation aligned with CXL specifications.
• Comprehensive Management
    ·   Typology view of CMM Devices, switch, and hosts
        o   CXL Memory Allocation/Deallocation with direct and pooling mode
        o   Host memory usage and statistical visualization

CapEx (TCO) Reduction: Consider a per node capacity increase of ~8TB using CXL in addition to existing DRAM capacity. In order to satisfy such a requirement using just DRAM would typically require to add additional physical server nodes, which not only adds unneeded cost of compute and peripheral devices but also management of additional nodes. CMM-B with Cognos helps reduce memory expansion cost by ~30% (based on Samsung internal device costing), while SCMC provides the needed management convenience through its composable memory architecture.

4. Practical Use Cases

Samsung’s CMM-B and SCMC solutions address a wide range of industry needs including:

1) IMDG Databases & Caches

• Maximizes network bandwidth utilization between compute and IMDG data nodes due to more data stored per node.
• SCMC data node agents can be configured to move data between CXL and DRAM such that maximum accesses are served from DRAM, thus, reducing data access tail latencies.

2) AI/ML Workloads

• Larger memory capacity allows additional data storage per node, allowing larger data transfers between data and compute nodes.
• Potentially accelerates machine learning training times, as a compute node can now fetch more data from one data node instead of several. Thus, reducing overall data transfer time by potentially saturating network bandwidths.
• To provide an additional performance boost, as depicted by the picture in section 2 above, SCMC agents running on data nodes can be configured (over REST) to profile a given application memory and move data such that maximum number of application data accesses are served by the fastest memory tier (DRAM, considering CXL as slower tier).
• Additionally, near memory compute workloads can now be able to access wider data range locally.

3) Financial Services

• Pub/Sub (Publish/Subscribe Messaging), using shared memory across multiple nodes
• Prevents out-of-memory conditions during trading peaks
• Keep extended—weeks or months—datasets in-memory for analytics
• Reduces cloud egress costs

Samsung Semiconductor’s CXL Memory Appliance comprised of CMM-B and SCMC is a game-changer for businesses navigating the challenges of AI-driven workloads. By delivering scalable, efficient, and flexible memory solutions, Samsung empowers companies to innovate, reduce costs, and deliver exceptional value to their customers. Explore the possibilities with Samsung’s CXL solutions and redefine what’s possible in memory technology.

For more information, please visit:
·      Samsung Cognos Management Console Whitepaper
·      CMM-B_Whitepaper

To engage further, reach out at:
·      Samsung Semiconductor Data Fabric Solution, MSL