Exploration and Exploitation for Buffer-Controlled HDD-Writes for SSD-HDD Hybrid Storage Server

Abstract

Hybrid storage servers combining solid-state drives (SSDs) and hard-drive disks (HDDs) provide cost-effectiveness and μs-level responsiveness for applications. However, observations from cloud storage system Pangu manifest that HDDs are often underutilized while SSDs are overused, especially under intensive writes. It leads to fast wear-out and high tail latency to SSDs. On the other hand, our experimental study reveals that a series of sequential and continuous writes to HDDs exhibit a periodic, staircase-shaped pattern of write latency, i.e., low (e.g., 35 μs), middle (e.g., 55 μs), and high latency (e.g., 12 ms), resulting from buffered writes within HDD’s controller. It inspires us to explore and exploit the potential μs-level IO delay of HDDs to absorb excessive SSD writes without performance degradation. We first build an HDD writing model for describing the staircase behavior and design a profiling process to initialize and dynamically recalibrate the model parameters. Then, we propose a Buffer-Controlled Write approach (BCW) to proactively control buffered writes so that low- and mid-latency periods are scheduled with application data and high-latency periods are filled with padded data. Leveraging BCW, we design a mixed IO scheduler (MIOS) to adaptively steer incoming data to SSDs and HDDs. A multi-HDD scheduling is further designed to minimize HDD-write latency. We perform extensive evaluations under production workloads and benchmarks. The results show that MIOS removes up to 93% amount of data written to SSDs, reduces average and 99 th -percentile latencies of the hybrid server by 65% and 85%, respectively.

An In-Depth Comparative Analysis of Cloud Block Storage Workloads: Findings and Implications

“…We also make similar observations in the I/O traffic sizes, in which AliCloud and TencentCloud are write-dominant (Figures 5(d)-5(f)). One possible reason of the write dominance in both AliCloud and TencentCloud is the wide use of application-level read caches in cloud storage, in which reads are mostly absorbed in the application layer without being issued to the storage layer [45]. Finding A.5: In AliCloud, larger volumes tend to have larger percentages of the total WSS over the raw capacity.…”

“…Modern cloud environments often host much more diverse types of applications, some of which feature unique characteristics (e.g., short-lived tasks [33]) that are not commonly found in traditional data center environments. Also, the workloads in MSRC are generally read-dominant [34], while the workloads in cloud environments are often write-dominant due to the heavy use of read caches in cloud applications [27,45]. Such mismatches motivate the need of collecting and analyzing comprehensive block-level I/O traces from real-world cloud block storage systems in large-scale production.…”

Levy Equilibrium Optimizer algorithm for the DNA storage code set

“…Wang et al [ 17 ] designed an encoding method consisting of repeated additive codes (RA) and an efficient hybrid mapping scheme to achieve a storage density of 1.67bit/nt. Zhang et al used combinatorial constraints to screen DNA storage codes [ 18 – 20 ], and used heuristic algorithms such as CLGBO [ 21 ] and NOL-HHO [ 20 ] to construct DNA storage code sets, constructing DNA storage code sets of higher quality. Yehezkeli et al [ 22 ] consider noise introduced entirely by uniformly repeated sequences and exploit the relation with equal weight integers in the Manhattan metric.…”

Optimization of Query Processing on Multi-tiered Persistent Storage

“…Financial and data processing requirements lead to the simultaneous utilization of persistent storage devices of different speeds and capacities. Therefore, data is distributed over many different storage devices with various capacity and speed characteristics [3]. This leads to a logical model of the multi-tiered organization of persistent storage [4,5].…”

“…Typically, the higher tiers of persistent storage have lower capacities and faster access times than the lower tiers. In a physical model, a sample multi-tiered persistent storage consists of the physical persistent storage devices available on the market, such as NVMe, SSD, and HDD [3].…”