Optimizing the SSD Burst Buffer by Traffic Detection

Shi, Xuanhua; Liu, Wei; He, Ligang; Jin, Hai; Li, Ming; Chen, Yong

doi:10.1145/3377705

Cited by 13 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use Equation (1) to calculate the bandwidth the application may obtain in different storage devices. In the work of Shi et al, 17 the I/O bandwidth and the random percentage of applications shows an inverse relationship, where, as the random percentage increases, the I/O bandwidth gradually decreases. To this end, when we have got the peak performance without I/O interference and the random percentage, the aggregate bandwidth B obtained by the applications which run concurrently on HDD under hybrid storage architecture can be calculated.…”

Section: Bandwidth Allocationmentioning

confidence: 95%

See 1 more Smart Citation

DDL‐QoS: A dynamic I/O scheduling strategy of QoS for HPC applications

Yang

Shi

Liu

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

With the increasing cloud-trend of high-performance computing (HPC), more users submit their applications simultaneously to the platform and wish they could finish before the deadline. Moreover, due to the severe holistic performance degradation caused by I/O contention, a deadline-sensitive I/O scheduler is needed to allocate storage resources according to the requirements of applications and resultantly guarantee the quality of service (QoS) of concurrently running applications. In this paper, we first explore the bandwidth allocation phenomenon caused by interference in applications through the modeling of historical data, and then we quote a metric called random percentage that can represent the random degree of the applications and be used to guide I/O scheduling in the later stage. We design a dynamic I/O scheduler named DDL-QoS that uses solid state drives(SSDs) as QoS guarantee to minimize interference and ensure applications meet their deadline. The potential of our design is that the greater the I/O interference, the greater the performance improvement, but this performance improvement will be limited by the physical properties of the storage hardware. KEYWORDS Deadline, I/O Scheduler, QoS 1 INTRODUCTION Nowadays, high-performance computing (HPC) systems are beginning to enter the exascale era. 1 IBM Summit, the fastest supercomputer in the world used at Oak Ridge National Laboratory is capable of 200 PetaFlops. 2 Sunway TaihuLight, the fastest supercomputer in China, has a peak performance of 120 PetaFlops. 3 The explosive growth of computing power requires the underlying parallel file system provides higher performance. At the same time, more and more data-intensive applications run on a large-scale in high-performance computing systems, resulting in an increasing demand for capable storage systems. Besides, some of the parallel file systems commonly used in HPC systems, such as Lustre, 4 GPFS, 5 OrangeFS 6 , etc, are starting to face significant challenges in terms of performance, complexity, and so on. 7 At the same time, HPC is migrating to the cloud as more and more HPC users begin to look to the cloud to help solve their workload challenges, and many public cloud companies have launched HPC products, such as Amazon Web Services, 8 Alibaba Cloud Computing, 9 etc. It means that storage resources in HPC system are shared between more and more different users. With the development of scientific applications, the scale of computing is growing. More and more storage resources are needed. Limited storage resources need to serve more applications. When multiple applications access the storage service concurrently, they will compete for I/O resources, which will lead to a serious drop in I/O aggregate bandwidth. 10 In addition, their I/O requests with different I/O access modes mixed. The hard disk drives (HDDs) can handle the requests with continuous mode more efficiently, while the requests with random mode will cause more seek overhead resulting in overall performance degradation. 11 We call this I/O i...

show abstract

Section: Bandwidth Allocationmentioning

confidence: 95%

“…The random factor is the methodology used in the work of Shi et al, 17 and is designed to indicate the frequency of disk head movement. First, sort the offsets of I/O requests.…”

Section: Architecturementioning

confidence: 99%

DDL‐QoS: A dynamic I/O scheduling strategy of QoS for HPC applications

Yang

Shi

Liu

et al. 2019

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…In the tests, we set the cache size varying from 16MB to 64MB by referring to [38]. Note that, the conigured cache space is dedicated for the write data, and the mapping table is separately saved 1 . Figures 14 and 15 show the overall I/O time (i.e.…”

Section: Case Study On Dram-equipped Configurationsmentioning

confidence: 99%

“…The NAND-based solid-state drivers (SSDs) have been widely employed in consumer devices and high performance computing platforms, thanks to their features of random access, lower power consumption, and collectively massive parallelism [1,2]. Though early SSDs were prohibitively expensive, the emergence of Multi-Level Cell (MLC), Triple-Level Cell (TLC) , and even Quad-Level cell (QLC) technologies has signiicantly driven down the per-unit price of SSDs with keeping multiple bits in a NAND cell [3ś5].…”

Section: Introductionmentioning

confidence: 99%

Degraded Mode-benefited I/O Scheduling to Ensure I/O Responsiveness in RAID-enabled SSDs

Sha

Cai

et al. 2022

ACM Trans. Des. Autom. Electron. Syst.

View full text Add to dashboard Cite

RAID-enabled SSDs commonly have unbalanced I/O workloads on their components (e.g. SSD channels), as the data/parity chunks in the same stripe may have varied access frequency, which greatly impacts I/O responsiveness. This paper proposes a I/O scheduling scheme by resorting to the degraded read mode and the read-modify-write mode, to reduce the long-tail latency of I/O requests in RAID-enabled SSDs. The basic idea is to avoid scheduling read or update requests to the heavily congested but targeted RAID components. Such requests are satisfied by accessing other relevant RAID components by certain XOR computations (we call the degraded modes ). Specially, we build a queuing overhead assessment model on the top of factors of data redundancy and the current blocked I/O traffics on SSD channels, to precisely dispatch incoming I/O requests to be fulfilled with the degraded mode or not. The trace-driven experiments illustrate that the proposed scheme can reduce the long-tail latency of read requests by 23.1% on average at the 99.99th percentile, in contrast to state-of-the-art scheduling methods.

show abstract

“…Based on parameters from previous studies [17,18], the total time consumed for reading and writing 4 KB are 14.2 ms and 15.1 ms, respectively, in HDD. The time consumed for reading and writing 4 KB are 25 µs and 230 µs in SSD, respectively [19].…”

Section: Introductionmentioning

confidence: 99%

Efficient Garbage Collection Algorithm for Low Latency SSD

Yuan

2022

Electronics

View full text Add to dashboard Cite

Solid-state drives (SSDs) are rapidly replacing hard disk drives (HDDs) in many applications owing to their numerous advantages such as higher speed, low power consumption, and small size. NAND flash memories, the memory devices used for SSDs, require garbage collection (GC) operations to reclaim wasted storage space due to obsolete data. The GC is the major source of performance degradation because it greatly increases the latency for SSDs. The latency for read or write operations is sometimes significantly long if the operations are requested by users while GC operations are in progress. Reducing the frequency of GC invocation while maintaining the storage space requirement may be an ideal solution to remedy this problem, but there is a minimal number of GC operations to reserve storage space. The other approach is to reduce the performance overhead due to GC rather than reducing GC frequency. In this paper, following the latter approach, we propose a new GC scheme that reduces GC overhead by intelligently controlling the priorities among read/write and GC operations. The experimental results show the proposed scheme consistently improve the overall latency for various workloads.

show abstract

Optimizing the SSD Burst Buffer by Traffic Detection

Cited by 13 publications

References 32 publications

DDL‐QoS: A dynamic I/O scheduling strategy of QoS for HPC applications

DDL‐QoS: A dynamic I/O scheduling strategy of QoS for HPC applications

Degraded Mode-benefited I/O Scheduling to Ensure I/O Responsiveness in RAID-enabled SSDs

Efficient Garbage Collection Algorithm for Low Latency SSD

Contact Info

Product

Resources

About