[Memory Tech Day 2023] Rethinking Storage Hierarchy for the AI Era

Samsung is preparing for the future of AI and Big Data. It is crucial to provide greater capacity, lower power consumption, extreme performance, and component quality to reduce the total cost of ownership (TCO) for Samsung customers. A series of presentations at the Samsung Memory Tech Day in October 2023 examined how Samsung is rethinking and reshaping the storage landscape. The first of these presentations came from Jay Hyun, Corporate VP of the NAND Product Planning Team at Samsung Electronics, and he illustrated the progress of moving more data processing to the “edge” of the compute system to reduce the number of random reads/writes and minimize fetching large quantities of data to the CPU and GPU. Coupled with edge processing, Samsung is enabling more data to be held and processed in cache memory, which will mean less data is transferred in and out of storage. The higher bandwidth speeds and memory caching will target lowering latency and provide more consistent Quality of Service (QoS) levels. Current VNAND solutions provide high capacity with greater bit density at higher IO speeds. The future goal of this technology is to expand from tripe-level-cell (TLC) VNAND to quad-level-cell (QLC) as more bit stacking is enabled on the chip. Each new generation is projected to increase the speed and density by multiples of 1.5 to 1.6x (the best in the industry). With these future implementations in place, Samsung will vastly improve bandwidth, decrease latency, and provide the means to ensure consistent QoS to Samsung’s customers. Leno Park, Vice President at Samsung Electronics, continued to expand on the vision of storage product improvements. Leno’s predictions forecast an increasing adoption rate of the QLC NAND technology for mobile devices, client PC SSD applications, and storage SSD devices for enterprise data centers. The forecast is that the newer QLC NAND devices will provide greater density in the memory of these devices. In turn, this will allow for more data to be held closer to the data processing layer. This will be vital as the AI trend requires more and more data to be moved back and forth between storage and processor. With the adoption of QLC NAND on the rise and the cost of QLC NAND declining, the QLC technology has the potential of providing future growth of NAND technology products such as SSDs and flash storage. The need for increased bit density on SSDs was explained by Jiho Cho, Corporate VP of the Flash Design Team at Samsung Electronics. The integration of QLC NAND with other existing NAND technologies currently in the market will enable devices to store greater amounts of data and reduce bit costs by improving the development of the NAND technologies. Cho and Park both agree that when combining NAND technologies in an “eco-system” (ECO NAND), the solution then provides greater capacities for flash storage and SSDs. The benefit of combining SLC, TLC, and QLC will be seen in the markets for mobile devices, client PC SSDs, and enterprise data center SSDs. This increased efficiency has the additional potential to eclipse current HDD market solutions where storage performance is not a critical factor. It will be well suited for data backup, long-tail multimedia content, and data storage. Efforts to reduce the TCO for future SSD products was outlined by Sarah Peach, Senior Director of the Product Management Team at the Americas Office of Samsung Electronics. To successfully hold market prominence, it is crucial that new technologies such as PCIe Gen6, QLC NAND and others lower the TCO. The formula for lowering TCO is driven by bit density in the SSD. The higher the bit density becomes, the smaller the SSD footprint becomes, which requires fewer storage racks in the data center. This in turn reduces the floor space required in the data center. CapEx is reduced when less space needs to be acquired. The increased density also means that the SSDs will need to process fewer operations to move data, and that will reduce energy consumption. There will also be fewer systems to maintain, which will reduce data center overhead. This will serve to reduce OpEx and bring down the overall TCO as a result. By offering various memory storage solutions, Samsung puts customers in a ready position for current and future trends.