A New Adaptive Coding Selection Method for Distributed Storage Systems

Wei, Bing; Xiao, Limin; Wei, Wei; Song, Yiheng; Zhou, Bingyu

doi:10.1109/access.2018.2801265

Cited by 7 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [25], LRC has been used to store warm data and to store cold data based on the access characteristics of the data in distributed storage systems. At the same time, an RS code is used to store cold data after optimizing for storage overhead to reduce the storage burden.…”

Section: Related Workmentioning

confidence: 99%

A New Minimize Matrix Computation Coding Method for Distributed Storage Systems

Yin

et al. 2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

With the number of nodes increasing in scale, the requirements of storage space enlarge sharply in distributed storage systems. Failure-tolerance schemes such as Reed–Solomon codes (RS codes in short) and Cauchy Reed–Solomon codes (CRS codes in short) are used to save storage space. However, these failure-tolerance schemes severely degrade the system performance. In this paper, we propose optimal RS codes (OptRS codes in short) based on RS codes and CRS codes that can offer better performance for encoding and decoding as well as maximizing the utilization of storage space. OptRS codes can speed up the matrix computation which is regarded as the most important factor to impact the efficiency of coding by transferring the matrix computation from the Galois field mapping to the XOR operation. OptRS codes employ an algorithm called row elimination scheme (RE scheme in short), which can eliminate the same XOR operation to minimize the number of XOR operations. We analyze optimal matrices (OM in short) in theory, which prove the optimal performance of OptRS codes over the Galois field. Our method is implemented on the top of the distributed storage system, and code parameters were carefully chosen. The test result shows that OptRS codes can improve the performance in different data block numbers, parity block numbers, block size, normal reading, and degraded reading, compared with RS codes and CRS codes.

show abstract

Section: Related Workmentioning

confidence: 99%

A New Minimize Matrix Computation Coding Method for Distributed Storage Systems

Yin

et al. 2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Furthermore, I/O access patterns are more complex when multiple applications with different workload characteristics are run simultaneously [13]. The "one-size-fits-all" solution cannot address this problem [13,[15][16][17]. In this paper, to address this problem, we present a new self-tuning client-side metadata prefetching scheme that uses multiple different prefetching strategies and dynamically adapts to workload changes.…”

Section: Introductionmentioning

confidence: 99%

A self-tuning client-side metadata prefetching scheme for wide area network file systems

Wei

Xiao

Song

et al. 2021

Sci. China Inf. Sci.

Self Cite

View full text Add to dashboard Cite

Client-side metadata prefetching is commonly used in wide area network (WAN) file systems because it can effectively hide network latency. However, most existing prefetching approaches do not meet the various prefetching requirements of multiple workloads. They are usually optimized for only one specific workload and have no or harmful effects on other workloads. In this paper, we present a new self-tuning client-side metadata prefetching scheme that uses two different prefetching strategies and dynamically adapts to workload changes. It uses a directory-directed prefetching strategy to prefetch the related file metadata in the same directory, and a correlation-directed prefetching strategy to prefetch the related file metadata accessed across directories. A novel self-tuning mechanism is proposed to efficiently convert the prefetching strategy between directory-directed and correlation-directed prefetching. Experimental results using real system traces show that the hit ratio of the client-side cache can be significantly improved by our self-tuning client-side prefetching. With regards to the multi-workload concurrency scenario, our approach improves the hit ratios for the no-prefetching, directory-directed prefetching, variant probability graph algorithm, variant apriori algorithm, and variant semantic distance algorithm by up to 15.

show abstract

“…Locally FR codes have lower disk I/O overhead compared with FR codes, but are not applicable for actual DSSs. Considering that the access of user to data is often unbalanced, that is, "hot" data is often accessed, "cold" data is rarely accessed [21,22], Li et al proposed variable fractional repetition (VFR) codes [23]. Although VFR codes take into account the imbalance of user access to data, the number of surviving nodes that need to be connected when repairing a failed node is equal to the number of data blocks in the failed node, not making the disk I/O overhead optimal.…”

Section: Introductionmentioning

confidence: 99%

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

Wang¹,

Wang²,

Wang³

et al. 2019

KSII TIIS

View full text Add to dashboard Cite

In order to improve the reliability and repair efficiency of distributed storage systems, minimum bandwidth regenerating (MBR) codes based on cyclic variable fractional repetition (VFR) codes are constructed in this thesis, which can repair failed nodes accurately. Specifically, in order to consider the imbalance of data accessed by the users, cyclic VFR codes are constructed according to that data with different heat degrees are copied in different repetition degrees. Moreover, we divide the storage nodes into groups, and construct MBR codes based on cyclic VFR codes to improve the file download speed. Performance analysis and simulation results show that, the repair locality of a single node failure is always 2 when MBR codes based on cyclic VFR codes are adopted in distributed storage systems, which is obviously superior to the traditional MBR codes. Compared with RS codes and simple regenerating codes, the proposed MBR codes based on cyclic VFR codes have lower repair locality, repair complexity and bandwidth overhead, as well as higher repair efficiency. Moreover, relative to FR codes, the MBR codes based on cyclic VFR codes can be applicable to more storage systems.

show abstract

A New Adaptive Coding Selection Method for Distributed Storage Systems

Cited by 7 publications

References 18 publications

A New Minimize Matrix Computation Coding Method for Distributed Storage Systems

A New Minimize Matrix Computation Coding Method for Distributed Storage Systems

A self-tuning client-side metadata prefetching scheme for wide area network file systems

Minimum Bandwidth Regenerating Codes Based on Cyclic VFR Codes

Contact Info

Product

Resources

About