Back in December of 2022, a new NVMe specification named ‘Flexible Data Placement’ or FDP was published with the potential to dramatically improve storage TCO for the world’s biggest hyperscalers. The new specification was the result of collaboration between Meta and Google, who had been independently working on solving the same problem: how to reduce or eliminate write amplification (WA) and avoid overprovisioning SSDs, a long-time industry ‘best practice’ for data centers. Developing FDP together was a strategic move since hyperscalers, OEMs and SSD manufacturers (including Samsung) would need a common standard in order to move forward. Since being ratified by NVMe Express®, FDP has been a hot topic at Flash Memory
Summit (FMS) and Open Compute Project Summit (OCP)
and while it’s arguably the most exciting data storage innovation to come along in years, it’s not yet widely known.
But before we dive into FDP, how it works, and what it means to hyperscalers, I should disclose that I was the lead author for the NVMe® TP4146a Flexible Data Placement technical proposal, which you can find here
. I worked with Meta, Google, and other NVMe members to combine efforts and develop a single standard. Getting the FDP specification ratified has been a story of innovation and collaboration between the biggest players in the industry (for the benefit of everyone) and for me personally, one of the highlights of my career at Samsung and as an active member of NVM Express. At FMS 2023, having Samsung demonstrating an SSD with FDP and FMS awarding NVM Express the Best in Show Most Innovative Memory Technology for FDP shows how collaboration turns ideas to reality.
FDP aims to solve a very expensive top-level problem: How to reduce CapEx and OpEx, and improve QoS for hyperscale datacenter SSD deployments. It’s not difficult to grasp why hyperscalers need better storage architecture solutions more than ever with the rise of AI, cloud applications, and media rich content. We humans are now producing zettabytes of data every week while our storage tech struggles to keep up.1
Disaggregated storage infrastructures separate the server's storage and compute resources. This allows for independent scaling and optimization of storage capacity and performance based on workload requirements. However, the standard disaggregated storage model isn't optimized to handle multiple applications running simultaneously and the vastly different workloads that ‘come and go’ and change over time. As a result, device performance remains unstable, which is inefficient and of course, costly.
To understand how an SSD with FDP outperforms a conventional SSD, let's consider how Write Amplification (WA) occurs in a standard disaggregated storage environment. WA is the additional writing of host data to media after that data is initially written by the host to media. This additional writing of host data is undesirable for several reasons: it requires additional storage capacity, it causes non-host induced wear, and the additional media reads and writes require more power and hurt overall system performance. Hyperscalers have to accommodate this extra data by overprovisioning. Write Amplification Factor (WAF) is the ratio of additional data that gets written every time the host sends data to the storage device. In a typical example, a WAF ~2.5 means that for every KB of data that gets written to the device, an additional 1.5 KB of incidental data gets written.
A system with FDP enabled gets around this by sharing information with the host about where
to allocate data on the media and the host may re-write that data or de-allocate that data before the SSD is required to move it which reduces the need for overprovisioning. To understand how an SSD with FDP outperforms a conventional SSD, let's consider how Write Amplification (WA) works in a standard disaggregated storage environment. With FDP, a WAF ~1 is now possible, and the implications for hyperscalers are, simply put, huge.
According to an OCP presentation
by Christopher Sabol from Google,
"Writes are one of the more power-intensive parts of a SSDs operations, doing 2 1/2 what you need pushes you up against that power envelope much more quickly."
For a more technical description of how FDP works, watch “Flexible Data Placement using NVM Express® Perspective” from OCP Summit 2022 here.
Flexible Data Placement (FDP) Use Case
Let’s compare how FDP and a typical disaggregated storage model each handle data distribution across the media for multiple applications. In this example, mixed data from applications A, B and C is written to available ‘super blocks’ across the media. Application A then de-allocates all of its data which must be garbage collected. Once complete, WAF is measured for both models.