Exceptional Scalability with CXL Memory: Samsung and Red Hat Expand the Ecosystem

In December 2023, Samsung Electronics announced that, for the first time in the industry, it had successfully verified CMM-D¹ memory operation with Red Hat, the world’s top enterprise Linux provider. The successful verification of CMM-D operations marks a significant step forward as technology companies collaborate on software technology for next-generation memory. On the back of its enhanced stability and strong security, Red Hat Enterprise Linux (RHEL) has earned a reputation as a highly trusted operating system (OS) for enterprises, based on an open-source ecosystem. In fact, more than 90% of Fortune 500 companies have worked with Red Hat.² This is in large part due to Red Hat’s strategy of updating and developing its OS in an open, easy-to-deploy format — instead of the traditional approach of applying for patents. The result is that Red Hat has established itself as a leader in open-source software optimized for business. As Samsung Memory and Red Hat continue to pursue parallel innovations in software and hardware, let’s take a look at the journey of their collaboration and the implications of the success it has brought. Samsung’s Next-Generation Memory Paired With Red Hat Software

Samsung Electronics currently has 13 memory solutions registered in the Red Hat Ecosystem, which supports a wide range of hardware for enterprise servers. Registered products are compatible with Linux and offer high-level security and comprehensive support. In addition, opportunities exist for Samsung Memory to create business with corporate customers that are using RHEL. These potential customers can also benefit from Red Hat’s technical support for a lifecycle of up to 10 years. In May 2022, Samsung Electronics and Red Hat took their collaboration to the next level by signing a Memorandum of Understanding (MoU) to integrate software with memory hardware and build a scalable ecosystem.³

Through this collaboration, Samsung and Red Hat were able to successfully verify CXL memory operations in RHEL 9.3. The verification was completed at the Samsung Memory Research Center (SMRC) and will lead the expansion of the CXL ecosystem — as well as create various business opportunities. Red Hat Enterprise Linux: Taking Full Advantage of CMM-D Released in November 2023, Red Hat Enterprise Linux 9.3 offers CXL drivers that support CXL Type 3 devices, which are also known as Memory Expanders. These drivers include support for CMM-D.

What advantages does CMM-D offer in the AI era? CXL is a unified interface standard that connects CPUs and GPUs to memory using PCIe 5.0. Amid the global AI boom, it is drawing attention as a next-generation interface with enhanced scalability. This scalability positions CXL as a fast, optimal solution for effectively processing the exploding amount of data that has resulted from advancements in AI/ML.⁴ The existing DRAM DDR interface has limitations when it comes to expansion, due to the restrictions of DRAM-module capacity and the number of DDR slots. To deal with this, there have traditionally been two main methods to further expand memory. First, it is possible to configure multiple memory nodes that consist of both CPU and memory. Second, if the host runs out of memory, Remote Direct Memory Access (RDMA) can be used to utilize memory from network-connected memory nodes. However, these approaches have drawbacks. Configuring additional memory nodes increases costs by adding more CPUs and systems. RDMA, on the other hand, can decrease performance due to the software overhead of data transfer. CMM-D solves both these problems by reducing overhead and enabling DRAM to be mounted to the PCIe interface, where SSDs are usually connected. This allows it to significantly expand memory capacity without the introduction of additional systems. SMDK, the Software to Support Next-Generation Memory

Non-Uniform Memory Access (NUMA) is a method for efficiently managing memory. It resolves bottlenecking by separating the areas of memory that are accessible by each CPU. Linux allows NUMA to run on CXL memory, maximizing the utilization of CMM-D — even when it is not dedicated to a specific CPU. This support for devices that are connected to CXL memory is a recent advancement, as NUMA was previously limited to Dual In-line Memory Modules (DIMMs). Specifically, in a Linux environment with Kernel 5.12 or higher, the Memory Hotplug feature can be used to create a mapping table for the NUMA nodes and Direct Memory Access (DMA) areas. If EFI_MEMORY_SP⁵ is activated on a CXL-device-equipped system, the CXL device is configured in device Direct Access (DAX) mode, which enables Online Memory Hotplug. The ways Linux has arranged the memory can be viewed in the /proc/buddyinfo and /proc/zoneinfo files.⁶ Currently, active discussions regarding software to support and utilize next-generation memory such as CMM-D are underway. Since 2021, Samsung Electronics has developed SMDK, a full-stack software development tool for CXL-based heterogeneous memory pools, which it has been sharing in open-source form since March 2022. SMDK provides CXL memory users and applications with tools for software development and management, as well as numerous experimental memory tiering functions to enable CXL use cases.

More Optimized Data Processing With NUMA: Memory Tiering & Pooling

CMM-D is set as a CPU-less NUMA node, and can be used in AI/ML and 5G systems with high memory loads, allowing more efficient and optimized data processing. Another advantage it brings is that existing functions can be used without change. □ Memory Tiering (Page Demotion) When seldom-accessed data is stored in local memory, it reduces the performance of frequently used (or “hot”) memory. Fortunately, this problem can now be effectively solved by using CXL memory with RHEL’s memory tiering function. This function allocates hot memory to the local memory tier and infrequently-used memory to the CXL tier, allowing for data migration between tiers when necessary.

□ Zero-CPU NUMA Hypervisor (KVM) In a large-scale cloud environment, service expansion has limits, as local memory is reserved for virtual machine (VM) instances. This limit can be overcome by allocating a VM to a CXL memory area and comprising an optimized infrastructure through resource scale-up. The diagram below shows an example of CXL NUMA memory being allocated to a KVM environment.

□ Podman and Containers As applications in existing VMs move on to container environments, nodes are expanding on a large scale due to the limitations of local memory. To overcome these limitations, containers can be allocated to the CXL memory area, while scale-up has the potential to be useful in configuring infrastructure. The diagram below shows an example of CXL NUMA Memory being allocated to a Podman environment.

□ Utilizing Application Processes CXL memory expansion creates a Zero CPU NUMA zone with normal and movable properties. The memory is allocated to the kernel in the same form and can be utilized in applications when necessary, through NUMA binding technology. When CXL memory is expanded, a zone in the form of Zero CPU NUMA is created. Then, when CXL memory is allocated to the kernel, it functions as a Zero CPU NUMA zone with normal and movable properties. This allocated CXL memory can be utilized in applications when necessary, through NUMA binding technology.

□ Configuring Huge Pages The kernel tries to make the most efficient use of memory by exchanging pages from memory to disk — or caching disk within memory. This works well in most environments, but when utilizing a database application, users may experience issues like high “iowait” or “unresponsive server” due to items having to wait in storage. These issues occur due to the high disak access count resulting from memory pages being replaced or that are about to be replaced while the application is trying to access data. Using huge pages can secure plenty of space that is pre-allocated for the application to use. This memory is not swapped in and out on disk and partially alleviates the inefficiencies seen in the previous environment. It also provides other benefits, such as reduced overhead in memory management.

□ Setting Up RAM Disks Files stored on a RAM disk partition can access data faster than a hard drive, because memory will generally always have much higher operating speeds than a disk. This is especially useful in cases where continuous access to some data helps increase performance, such as using a RAM disk partition to provide quick access to web server data.

Distributing the CXL Memory Utilization Guide With the successful verification of CMM-D, datacenter customers are now able to use Samsung’s next-generation memory without additional software changes, for improved ease of use. Looking to the future, Samsung plans to expand its collaboration with software, server, and chipset vendors. It will also distribute a guide document that makes it easy to install and utilize CMM-D on RHEL, which will aid the commercialization of next-generation memory. This document will be available on the Red Hat website. From a broader perspective, this software collaboration is of major significance to the memory industry because its positive outcomes will serve to broadly expand the base of CXL-based memory products — as commented on below by key experts from Samsung and Red Hat. “Changing memory management for CMM-D requires constant discussion,” said Taehoon Lee, Strategic Account Manager at Red Hat. “We need to consider using CMM-D for 5G Network applications like 5G vDU and 5G UPF, where huge pages are essential, or for applications with high memory demands, such as AI/ML. “Until communities like Kernel.org reach a consensus on the optimal architecture for CMM-D, we need a way to manage memory tiering in applications. We also need to discuss disaggregated computing, which can be used on cloud platforms such as OpenStack or Openshift," Lee emphasized. "We are working closely with SMRC to provide support at the IaaS and PaaS⁸ levels." "Red Hat is a well-established leader in the Linux industry, and we believe this first collaboration on CXL Enablement will significantly contribute to the development of our memory product ecosystem," said Kyungsan Kim, Principle Engineer of DRAM Solution Team at Samsung Electronics. "We look forward to more close collaboration between our two companies resulting in the development of integrated system solutions based on open source principles." "The collaboration between Samsung and Red Hat to enable CMM-D is just the beginning of a long, rewarding journey," said Byungsoo Kim, Director at Samsung Memory Research Center (SMRC), who oversaw the project. “We hope that not only operating system companies, but also a wide array of partners — in areas ranging from servers to devices, platforms and applications — will join us in expanding the CXL ecosystem. SMRC will provide an optimized open platform to achieve this." Currently, the two companies are preparing to collaborate on building a reference architecture and creating an implementable model for distributed memory, in which all computing resources (CPU, GPU, and Memory) are connected by a fast network. The collaboration will include participation in various forums and technical activities from both Samsung and Red Hat. At the same time, the Samsung Memory Research Center (SMRC) will continue its efforts to expand the CXL ecosystem and provide solutions that push the next generation of memory even further.

---

_{¹ Compute Express Link™ (CXL™) Memory Module DRAM. ² https://www.redhat.com/en/about/company ³ https://news.samsung.com/global/samsung-electronics-and-red-hat-announce-collaboration-in-the-field-of-next-generation-memory-software ⁴ “AI/ML” stands for “artificial intelligence and machine learning.” Although these technologies are not entirely the same, they are often grouped together. ⁵ The EFI_MEMORY_SP attribute bit refers to memory that is reserved for a specific purpose. ⁶ https://access.redhat.com/solutions/700683 ⁷ https://ieeexplore.ieee.org/document/10016720 ⁸ “IaaS” and “PaaS” stand for “infrastructure-as-a-service” and “platform-as-a-service,” respectively.}