[Memory Tech Day 2023] Memory for the World of Tomorrow

In the grand scheme of things, you wouldn’t think a tiny memory device would be so important in the global landscape. Yet it turns out to be at the crux of a world undergoing rapid change, revolving around data centers. The Samsung Memory Tech Day held in San Jose last year took the opportunity to elaborate on this and other topics. The following recaps the presentations from Track 6 of the Tech Day, “System Level Collaboration & New Opportunities.” Presenters told the story of how incredible improvements in DRAM and flash storage have enabled data centers at the heart of the world’s global commerce to implement a dynamic cloud. UPCIe 6.0 improves on PCIe 5.0 and is made possible by the collaboration between design and IP venders to bring new projects to life. But the story wouldn’t be complete without a discussion of the elephant in the room: how can such growth in cloud capacity and capabilities be met without increasing energy consumption and natural resource depletion? Intelligently and thoughtfully designed technology actually helps reverse over a century of escalating carbon emissions. The world is expecting big things out of a tiny memory device. Let’s take a peek at those expectations and Samsung’s part in them. In the world of tomorrow, the companies building the data centers using these memory devices will:

• • Design their servers through a collaborative super network of support partners, dedicated to rapid deployment of even-more-sophisticated platforms
  • Test their designs and theories in specialized labs that allow them to make technology leaps without incurring risks and expenses

Once in operation, the data centers themselves will:

• • Run the memory devices faster and hotter than ever before by operating deep in the world’s oceans to take advantage of free cooling
  • Demand more performance and capacity from the devices, yet expect lower power consumption
  • Cycle through an enormous number of these devices throughout the server lifecycle

The users of these data centers accessing them as “the cloud” will:

• • Range from giant multinational conglomerates to tiny startup businesses, all expecting customized handling of their particular needs
  • Have the option to operate out of public or private clouds that they select, balancing ease of scaling, cost, and security

As the company at the forefront of technology for these devices, Samsung is acutely aware of its responsibilities to ensure its future technology plans and environmental stewardship dovetail elegantly to the perfect memory products of tomorrow for a better world in the future. Session 1 Hyung-Jin Kim, leading the Cloud Platform group, started off the sessions by discussing the merits of public vs. private cloud and why Samsung has chosen to take its cloud in-house. And there’s plenty of cloud business to go around; in the last 10 years, the cloud business has grown around 20% each year. Public Clouds A variety of public clouds are popular choices, and for good reasons:

• • The pay-as-you-go-model is pervasive with servers ready to instantiate appropriate numbers of virtualized CPUs to handle dynamically changing workloads
  • Hardware used to implement public cloud data centers is of a standardized but modular design (host processor, memory, and storage following OCP methodologies) to adapt to different needs and therefore are quite cost-effective
  • Total cost of ownership is reduced when shared resources can be scaled up massively
  • Sustainable resourcing can be boosted by employing things like more efficient energy methodologies and advanced cooling concepts that are practical only on a large scale, leading to a manageable carbon footprint and greener IT
  • Well documented mechanisms and widely understood APIs make it easier to ramp up a new project in the cloud

However, not everything is ideal. Virtualized CPUs are implemented and managed by a Hypervisor. For tasks that are compute-intensive but fluctuating like compiling code, the Hypervisor will provision for the maximum demand; an overprovisioning ratio of 200% is not uncommon in these cases. Therefore, the virtualized approach can become expensive - compared to dedicated bare metal resources. Samsung In-House Cloud Samsung chose to develop an in-house cloud for good reasons as well.

• • National security: valuable technologies may be best protected when they never leave the private cloud
  • Optimization of key operations: each manufacturing step requires its own local IT optimization (tight-knit communication between the manufacturing and IT groups), which is much easier to control in an in-house environment
  • Scale: Samsung requires vast amounts of cloud computing capability and storage capacity, easily justifying its own private cloud
  • Innovation: having its own private cloud allows Samsung to experiment and develop leading edge technologies, providing a product test bed for reference architectures, hypothetical use cases, and accelerated development

Session 2 In session 2, Myoungbo Kwak, Master of IP Development at Samsung Electronics, presented on the topic of Memory IP and EDA, and highlighted the importance of collaborative efforts to meet the technology demands of today and in the future. PCle 5.0 Samsung’s PM1743 has the fastest speed in SSD; board design is key to mitigate:

• • High speed and channel performance impediments
  • Insertion loss, reflection loss, and crosstalk
  • IO margin failure and system function issues

To meet these demands, Samsung supports customers’ board designs through:

• • Channel Analysis
  - TDR (Zo, Reflection)
  - Full channel insertion, return loss, and crosstalk
• • Channel Optimization
  - Differential via design
  - Minimize discontinuity points
• • OMT Prediction
  - IO margin simulation
  - Correlation between simulation and test
Samsung also supports Early Power Integrity Optimization due to increased needs in set level DRAM PI analysis, the need for low power at high speed, DRAM current tendency to increase, and the difficulty of set power distribution network design. The process works like this:
• • Set maker customer provides system info
  - System PCB Design, Cap, and PMIC model
• • Samsung supports set level analysis
  - DC IR-drop & Power Network Impedance
  - Power Network Design Optimization
  - Power Noise Level Prediction
The result is enhanced set level system PI performance with an IR-drop improvement, averaging 25%. PCle 6.0 Storage interfaces are moving from a parallel digital interface in eMMC to an LVDS interface in Universal Flash Storage (UFS), a must-have technology for both 5G mobile and automotive applications. Mobile applications need PAM4 signaling due to insertion loss.
• • With PCIe 6.0, the signaling interface moves from NRZ two-level signaling to PAM4 four-level signaling as used in Gig Ethernet; keeping the data eye open demands different approaches to board design
  • Clock and data recovery logic becomes significantly more of a digital task in PCIe 6.0, with the focus moving to latency in the clock recovery loop as the limiting factor in interface speed
  
  
  

Samsung is promoting PCIe 6.0 IP Early Joint Validation. Early testing of the protocol level with the partner’s test chip boosts a time-to-market advantage. Through its 2.5D/3DIC design platform on a single canvas initiative, Samsung and its partners collaborate to provide customers with pre-verified workflows:

• • Native 3D Flow
  • System Planning
  • Analysis (SI/PI/Thermal/IR-drop)

Additionally, Samsung collaborates with EDA partners to create an open platform value chain in cooperation with our customers. We call this 3-way open innovation. Samsung developed this open platform using design standards, which combine state-of-the-art tools, design flow, and design methodologies. Finishing out the session, Mike McSweeney from Synopsys provided an overview of how Synopsis takes part in memory IP early validation collaboration. Session 3 During session 3 of the Memory Tech Day, Yvette Lee, Ph.D., Program Manager of the Memory Product Planning Team at Samsung Electronics, spoke about the Samsung Memory Research Center (SMRC). In a nutshell, SMRC is a collaboration platform where customers and partners are welcome to use the small data center facility to test and prove ideas. The SMRC provides component, system, software, and application experts with a 24-server rack capacity lab, running a 400Gb internal service network for testing and verification purposes. What could you achieve with a playground where you could try out advanced technologies like these?

• • Memory-Semantic SSD: utilizes CXL instead of being addressed as a standard block device
  • Smart SSD: with processing technology to reduce CPU utilization and data transfer bottlenecks
  • FDP SSC (QLC based SSD): using NVMe Zoned Namespaces (ZNS) to expose a zoned block storage interface between the host and SSD that aligns the data to its media perfectly
  • Ultra-High-Density Storage: a petabyte-scale storage solution that increases rack-scale space efficiency
  • Storage devices implementing the Samsung Telemetry Analysis Platform: for device health checks and device failure predictions

The lab can compare device performance under different platforms such as Redhat’s Openstack (IaaS)/Openshift (PaaS), VMWare vCenter/Tanzu, and bare metal; the OS can be CentOS/Ubuntu, RHEL, or others. Add your favorite databases for analysis: Oracle DB, Maria DB, MongoDB, commonly used in HPC and AI/ML/Big Data applications, MYSQL, and PostgreSQL.

You can then mix and match current and upcoming memory and storage technologies: - CMM-D (CXL Memory Module for DRAM) - CMM-H/HC (Hybrid/Hybrid Compute) - FDP (Flexible Data Placement) - CSD (Computational Storage Device) - NPU (Neural Processing Unit) - DPU (Data Processing Unit)

In the second half of the session, Isaac Choi of the SMRC lab in San Jose presented four collaborative efforts to demonstrate the power of the SMRC facility.

Session 4 For session 4, presenters Jang-Seok Choi and Sam-Jong Choi shared Samsung’s vision of the carbon-neutral world to come, and how it will demand a “circular economy” where products will not only be recycled but also will sometimes be “upcycled” to better take advantage of their intrinsic value. This vision revolves around the Environmental, Social, and Corporate Governance (ESG) initiatives being undertaken worldwide, focusing on the processes and procedures Samsung has put into place to reduce not only its own carbon emissions but also those of its customers. In the first half of the session, Jang-Seok Choi, VP of the New Business Planning Team at Samsung Electronics, provided an overview of the recertification of DRAM and SSD products as yet another means to achieve carbon neutrality. In the second half, Sam-Jong Choi, Ph.D., who leads the Material Technology Team at Samsung Electronics, provided an overview of ESG from a materials perspective and offered Samsung-specific examples:

• • The move to new industrial gases with a significantly reduced Global Warming Potential (GWP)
  • New Reduce Resist Coat (RRC) technology that uses significantly less photoresist solutions to achieve the same high-quality results
  • New gas recovery technology to capture and reuse it in the equipment and supply-unit
  • Reuse of wafer transport containers

In summary, this presentation track showed us the possibilities for a brighter future evolving from the exceptional memory research and planning being done at Samsung today.