Efficient Scheduling for Multi-Block Updates in Erasure Coding Based Storage Systems

Shen, Jiajie; Zhang, Kai; Gu, Jiazhen; Zhou, Yangfan; Wang, Xin

doi:10.1109/tc.2017.2769051

Cited by 19 publications

(15 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These approaches are devoted to accelerating IO access while updating. Besides, Shen et al [50] proposed UCODR in 2017, which is a heuristic of scheduling. They defined the distance between 2 parity nodes and started to update from the closest nodes, which can reduce the number of disk read.…”

Section: B Resource Optimizationmentioning

confidence: 99%

“…Block needs to be read and written FIGURE 5: An example of RCW and RMW [50]. Assuming there are n = 7 nodes and k = 6 data nodes.…”

Section: B Data Transmission Approachesmentioning

confidence: 99%

“…What's more, the analysis [14] of MSR Cambridge traces [37] and Harvard NFS traces [21], infers updates are small, since most of the traces have more than 60% of updates smaller than 4KB. While Shen et al [50] have a different observation that update requests with large update sizes are quite common in existing distributed storage systems, especially for online applications [16], [32], [57]. These different observations show that DU is mutable, thus update schemes should be compatible with small updates and large updates.…”

Section: Introductionmentioning

confidence: 99%

“…According to these reasons (requirements) of DU in erasure coding, a variety of update schemes based on erasure coding have been proposed in the literature to optimize DU, including optimizing computation schedule [27], [41], traffic overhead reduction [39], [59], IO overhead reduction [50] and modern hardware acceleration [58]. All of these techniques were proposed individually previously.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Erasure Coding-Oriented Data Update for Cloud Storage: A Survey

2020

View full text Add to dashboard Cite

Erasure coding is the leading technique to achieve resilient redundancy in cloud storage systems. However, it introduces two prominent issues: data repair and data update. Compare to data repair, data update is much more common. A variety of update schemes based on erasure coding have been proposed in the literature to optimize data update, such as computation optimization, network traffic overhead reduction, IO overhead reduction, and modern hardware acceleration. However, all of these techniques were proposed individually previously. In this work, we seek to summarize them systematically and group them in a new form. First, we generalize the state-of-the-art researches and introduce existing classifications. Moreover, based on our observation, we propose two classifications: resource-based classification and tierbased classification. In resource-based classification, we group these techniques according to the resource they optimize and introduce them in detail. In tier-based classification, we propose a novel hybrid technique framework with five tiers and conduct a comprehensive comparison between these techniques. We make a conjecture that most techniques in different tiers can be used jointly. Finally, we conclude the research challenges and potential future works. INDEX TERMS data update, cloud storage, erasure coding, survey

show abstract

Section: B Resource Optimizationmentioning

confidence: 99%

“…Block needs to be read and written FIGURE 5: An example of RCW and RMW [50]. Assuming there are n = 7 nodes and k = 6 data nodes.…”

Section: B Data Transmission Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Erasure Coding-Oriented Data Update for Cloud Storage: A Survey

2020

View full text Add to dashboard Cite

show abstract

“…CAU [25] relocates updated data blocks to different nodes in the same rack during the additional submission iteration to reduce cross-rack update traffic. Shen et al [12] propose a heuristic update scheduling algorithm UCODR to quickly construct an update operation sequence and reduce the disk I/O overhead of multiple block updates. Hybrid-U [35] is a hybrid update scheme in-memory storage that minimizes network update I/O by executing update requests in parallel.…”

Section: Related Workmentioning

confidence: 99%

SDCUP: Software-Defined-Control Based Erasure-Coded Collaborative Data Update Mechanism

Chen

Zhong

2020

IEEE Access

View full text Add to dashboard Cite

The storage system often uses erasure coding to provide the necessary fault tolerance. The erasure-coded update involves data transmission and data calculation of multiple nodes. Frequent updates will cause massive communication overhead. This paper mainly considers two issues: (1) In the scenario of frequent small-size updates, there are repetitive behaviors upon update, which causes bandwidth consumption to increase exponentially as the number of update nodes increases; (2) with the increase of data scale, there are local link busy phenomena caused by unbalanced use of links during update, which can prone to bottleneck links. In order to improve the inefficient update due to network bottlenecks. We propose SDCUP, a software-defined-control collaborative update mechanism that reduce update time for erasure-coded data with network load balance. Specially, SDCUP uses the software-defined control method to select the update transmission path according to the actual link load and adjust the data flow transmission rate by monitoring the degree of network load balance periodically. To further reduce the cross-rack update traffic, SDCUP unloads the calculation to the switch to realize the data aggregation in the rack, and it parallelizes sub-update operations to efficiently and cooperatively update. To evaluate the performance of SDCUP, we conduct simulation experiments on Mininet with real-world traces. The simulation results show that SDCUP can achieve a better load balance in multiple scenarios. Compared with the other data update schemes, the proposed method can improve the system throughput by up to 21% and reduce the update time by up to 47%. INDEX TERMS Erasure coding, software-defined, data update, network load balance.

show abstract