Adaptive Bandwidth-Efficient Recovery Techniques in Erasure-Coded Cloud Storage

Nachiappan, Rekha; Javadi, Bahman; Calheiros, Rodrigo N.; Matawie, Kenan M

doi:10.1007/978-3-319-96983-1_23

Cited by 7 publications

(10 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Erasure code has lower storage overhead than replication, which are widely used in cloud systems [1][2][3] , there is various kind of erasure codes, RS code 4 is a traditional code and widely deployed in the storage system, it is because it can tolerate any m blocks are lost, m is the number of parity blocks of a stripe there. After the Google disclosed Google File System 5 , which used triple replication with the diffusion of commodity disks.…”

Section: Erasure Codementioning

confidence: 99%

Optimization of recovery bandwidth utilization in erasure code storage systems

xing

2022

Third International Conference on Computer Science and Communication Technology (ICCSCT 2022)

View full text Add to dashboard Cite

In the erasure code storage systems, failure is very common, so a great amount of data needs to be recovered in the system. For each recovery task, we need to select some nodes from the nodes of a stripe to read data to recover the failed data. However, due to the limitation of the bandwidth of each node, the number of simultaneous recovery tasks is limited, and the inappropriate task selection causes the waste of bandwidth, resulting in slow recovery. This paper proposes two greedy approaches Gre-N and Gre-T to solve the above problems. Gre-N assigns tasks to nodes with high bandwidth first. Gre-T sorts tasks in weighted order. The node with the lowest bandwidth utilization is selected each time to avoid the bandwidth exhaustion of a large number of nodes. In addition, the bandwidth utilization of the system is balanced by the low bandwidth node compensation policy. According to the experimental results, compared with the original approach, Gre-N and Gre-T increase the bandwidth utilization by up to 20.2% and 29.7%.

show abstract

Section: Erasure Codementioning

confidence: 99%

Optimization of recovery bandwidth utilization in erasure code storage systems

xing

2022

Third International Conference on Computer Science and Communication Technology (ICCSCT 2022)

View full text Add to dashboard Cite

show abstract

“…Compared with ACL, DFC, Pyramid and Xorbas, NEC can reduce the repair cost and repair time by 26.8%-40% and 13.9%-37.1% and HRepair further reduces the cost of repair time. In [43] authors proposed a hybrid method of redundancy and erasure code based on a proactive prediction approach to retrieve data from expected failure devices. The proposed approach applies slow recovery of cold data and copying of hot data to memory .Their results showed improvement in average recovery bandwidth.…”

Section: ) Recovery Bandwidth and Network Traffic Solutionsmentioning

confidence: 99%

A Survey of Solutions and Future Directions for the Challenges of Implementing Erasure Codes in Cloud and Fog Computing

Al-Shami¹,

Al-mutawkkil²

2020

ijarcs

View full text Add to dashboard Cite

More recently, computing storage systems have started implementing an erasure code approach in storing big data due to its many advantages such as reducing storage cost, providing fault tolerance, and reliability. However, many challenges and open issues associated with the use of a code erasure approach in cloud and fog computing environments have arisen. Different methods and techniques for solving these challenges have been presented by many recent studies, however, there are still shortcomings in comprehensive surveys. Therefore, this paper attempts to present a survey of the most important studies in the last five years and discuss the solutions and future trends in this field, in addition to providing a classification of these solutions according to the main research challenges and comparative analysis of the proposed solutions with respect to objectives, environments, tools, and codes used. The ultimate goal of this paper is to provide a summary for researchers for the development of more future work.

show abstract

“…In a preliminary version of this paper [21], we proposed a cloud storage system that employs several proposed bandwidth-efficient proactive recovery methods. In the event of any disk/machine failure prediction, the client's durability, availability, and performance requirements are determined using a Service Level Agreement (SLA).…”

Section: Introductionmentioning

confidence: 99%

“…However, it does not take into account important system parameters like data duplication and the system's current network traffic during proactive recovery that may incur additional storage overhead and network bandwidth/traffic. This paper is the extended version of our previous paper [21] and the differences between this work and previous work are:…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimized Proactive Recovery in Erasure-Coded Cloud Storage Systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

Cloud data centers have started utilizing erasure coding in large-scale storage systems to ensure high reliability with limited overhead compared to replication. However, data recovery in erasure coding incurs high network bandwidth consumption compared to replication. Cloud storage systems also play an important role in the energy consumption of data centers. Heuristic proactive recovery algorithms select all data blocks from failure-predicted disk/machine and perform proactive replication that contributes to huge recovery bandwidth savings. However, they fail to optimize the selection. Optimization can further improve resource savings. To address this issue, we propose a recovery algorithm that applies minimization on data blocks selected for proactive replication by considering the necessary and appropriate constraints that are constructed based on the system's current network traffic and data duplication information. We evaluate the proposed algorithm using extensive simulations. Experiments show that the recovery algorithm reduces network traffic by 60% and storage overhead by 46% compared to the heuristic proactive recovery approach. Also, the proposed proactive recovery methods reduce the storage system's energy consumption by up to 52% compared to replication.

show abstract

Adaptive Bandwidth-Efficient Recovery Techniques in Erasure-Coded Cloud Storage

Cited by 7 publications

References 14 publications

Optimization of recovery bandwidth utilization in erasure code storage systems

Optimization of recovery bandwidth utilization in erasure code storage systems

A Survey of Solutions and Future Directions for the Challenges of Implementing Erasure Codes in Cloud and Fog Computing

Optimized Proactive Recovery in Erasure-Coded Cloud Storage Systems

Contact Info

Product

Resources

About