A Real-Time, Low Latency, Key-Value Solution

Samsung Z-SSD Ultra-low Latency Storage In this whitepaper, we present a high performance key-value store solution that can be used in a wide variety of latency sensitive applications tied to the rapidly expanding era of hyper-speed “big data.” By combining Levyx’s high-performance Helium™ key-value store with Samsung’s ultra-low latency Z-SSD™, we have demonstrated performance improvements of up to 10X over the conventional approaches of processing large-scale datasets. Over the past decade, many “big data” application requirements have evolved from those of a batch-oriented nature, to being increasingly time sensitive and now all information is demanded in real time. Many real-time operations and corresponding data-intensive analytics have fanned out from their Social Media origins to encompass a broad range of areas including Internet of Things(IoT), financial trading, manufacturing, e-commerce, cybersecurity, utility grids, life sciences, automotive and more. While existing big data architectures have been able to keep up with overall IO throughput requirements by simply scaling out (by adding more CPUs, nodes, servers, SSDs, etc.), they are increasingly challenged by workloads where very low latency is especially critical. Samsung’s Z-SSDs deliver a dramatic improvement in latency—making them strikingly efficient solutions for large-scale, real-time environments. In particular, opportunities centered on IoT are gaining momentum, and today, the number of intelligent interconnected devices is far outstripping the amount of associated human interaction. Managing latency in these environments is particularly challenging as machine-to-machine speed demands are beginning to drive latency and jitter requirements from social media-acceptable levels of 10-100s of milliseconds down into 10s of microseconds, and even lower. The financial sector also requires low-latency solutions given that financial institutions are now predominantly automated and algorithmically-driven, where trades are executed in 100s of nanoseconds. The need for low-latency applied to large-scale data exists today in financial functions such as the back-testing of trading algorithms, compliance verification, risk analysis, and fraud detection. These capabilities must keep pace with the overall speed of transaction-intensive data mining and real-time analytics required in the world of finance. Samsung, the leading supplier of flash-based technologies, has developed an extensive portfolio of flash storage products that effectively address the increasing density, throughput and performance requirements of today’s big data applications. Lightweight protocols like NVMe can leverage the parallelism of NAND technology to deliver one million IOPS and 3GB/s throughput from a single storage device. At the same time, network hardware can deliver 100Gbps throughput and dramatically reduce latency, while challenging assumptions about data locality and disaggregated storage. Samsung has made major hardware advances to accommodate real-time, data-intensive applications to the extent that the conventional (non-optimized) software stack has become the single greatest source of latency and non-deterministic system performance. Levyx set out to resolve this and has re-engineered the data stack to fully extract the benefits of this new hardware, making the pairing of its Helium key-value store with Samsung’s Z-SSDs a compelling, low-latency and high-performance flash-based solution. About Levyx Levyx has developed next-generation database storage and query offload engines that fully exploit the latest commodity hardware technologies including multi-core servers, internal and external flash systems, and IO offload engines. The result is unprecedented performance and latency reductions for IO intensive workloads, such as financial service backtesting and streaming analytics. Levyx’s new software-based engines allow for first time, persistent computing to be possible on “big data” platforms such as Apache Spark thru the use of SSDs instead of volatile memory-only designs. Levyx is now delivering the world’s fastest key value store (Helium) and world’s first Distributed Storage/Analytics Offload Engines running on Flash (Xenon). Find out more about the pairing of Samsung Z-SSD with Levyx’s Helium in the whitepaper below.