Purity

Abstract

Although flash storage has largely replaced hard disks in consumer class devices, enterprise workloads pose unique challenges that have slowed adoption of flash in "performance tier" storage appliances. In this paper, we describe Purity, the foundation of Pure Storage's Flash Arrays, the first all-flash enterprise storage system to support compression, deduplication, and high-availability.Purity borrows techniques from modern database and keyvalue storage architectures, and introduces novel storage primitives that have wide applicability to data management systems. For instance, all writes in Purity are monotonic, and deletions are handled using an atomic predicate-based tuple elision primitive.Purity's redundancy mechanisms are optimized for SSD failure modes and performance characteristics, allowing for fast recovery from component failures and lower space overhead than the best hard disk systems. We built deduplication and data compression schemes atop these primitives.Flash changes storage capacity/performance tradeoffs: unlike disk-based systems, flash deployments are rarely performance bound. A single Purity appliance can provide over 7 GiB/s of throughput on 32 KiB random I/Os, even through multiple device failures, and while providing asynchronous off-site replication. Typical installations have 99.9% latencies under 1 ms, and production arrays average 5.4× data reduction and 99.999% availability.Purity takes advantage of storage performance increasing more rapidly than computational performance to build a simpler (with respect to engineering, installation, and management) scale-up storage appliance that supports hundreds of terabytes of highly-available, high-performance storage. The resulting performance and capacity supports many customer deployments of multiple applications, including scaleout and parallel systems, such as MongoDB and Oracle RAC, on a single Purity appliance.

Extending and Programming the NVMe I/O Determinism Interface for Flash Arrays

“…In terms of which layer tames the SSD performance issues, vast research has been done, from device-only modifications [10,26,44,[52][53][54], host-level changes [32,35,36,55,56], and transparent approaches on programmable devices [12,25,39,57,58] to interface solutions [18,51,[59][60][61]. Device-level proposals usually require vendors to significantly modify the firmware policies, which are not attractive for quick deployment; host-only optimizations can only guarantee a soft contract (i.e., not eliminating background interferences); transparent approaches do not work for commodity SSDs, and many interface-level solutions focus on various types of inefficiencies of the existing software/hardware stack.…”

“…It is important to compare IODA comprehensively, but because other works use varying platforms (some even cannot run), "apples-to-apples" comparison would be difficult to make. With our upgraded FEMU, we were able to re-implement state-ofthe-art techniques [26,30,32,34,36] in around 3,400 LOC. Here we provide more details on how we implement related work on the FEMU stack (the changes are either in the Linux kernel or FEMU):…”

“…• Proactive/Cloning [3,32]: Despite user I/O size, proactive methods [32] issue full-stripe reads to the array. It only waits for the first few returned I/Os, which are enough to reconstruct user data before marking the user I/O done and returning to upper layers.…”

“…Compared to an "ideal" scenario where there are no write-triggered GCs, IODA is only 1.0-3.3× slower between p95-p99.99, whereas the baseline suffers from 1.1-88.3× degradation. To compare IODA with state-of-the-art approaches, we also re-implement seven published methods that represent preemption [26][27][28], program/erase (P/E) suspension [29][30][31], speculation [32,33], GC coordination [34,35], partitioning [36][37][38], "tiny-tail" controller design [10], and SLO-aware prediction [39].…”

Identity of university Chinese heritage language learners in Hong Kong

“…We present ZapRAID, a high-performance RAID system for ZNS SSDs by carefully exploiting Zone Append to achieve high write performance via intra-zone parallelism; in the meantime, ZapRAID supports lightweight stripe management in terms of: (i) querying stripe metadata during degraded reads, crash recovery, and full-drive recovery, (ii) memory usage for indexing, and (iii) persistent storage for metadata. ZapRAID builds on log-structured RAID (Log-RAID) [25,28,34,41], which issues sequential writes to a RAID array, and extends Log-RAID with a novel group-based data layout. Specifically, ZapRAID partitions stripes into stripe groups and issues Zone Append to the stripes within the same stripe group for high write performance.…”