An Efficient Binary Locally Repairable Code for Hadoop Distributed File System

Shahabinejad, Mostafa; Khabbazian, Majid; Ardakani, Masoud

doi:10.1109/lcomm.2014.2332491

Cited by 48 publications

(15 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Compared to q-ary LRCs, BLRCs are known to be advantageous in terms of implementation in practical systems. In [43], the advantages of (n, k, d, r) = (15,10,4,6) BLRC are discussed and compared with (16,10,4,5) non-binary LRC, (14,10) RS code, and three-replication with four metrics including encoding complexity, repair complexity, mean time to data loss, and storage capacity. The authors of [43] further analyzed the advantages of BLRCs with a high Hamming distance and average locality [44,45].…”

Section: Binary Locally Repairable Codesmentioning

confidence: 99%

Overview of Binary Locally Repairable Codes for Distributed Storage Systems

Kim

2019

Electronics

View full text Add to dashboard Cite

This paper summarizes the details of recently proposed binary locally repairable codes (BLRCs) and their features. The construction of codes over a small alphabet size of symbols is of particular interest for efficient hardware implementation. Therefore, BLRCs are highly noteworthy because no multiplication is required during the encoding, decoding, and repair processes. We explain the various construction approaches of BLRCs such as cyclic code based, bipartite graph based, anticode based, partial spread based, and generalized Hamming code based techniques. We also describe code generation methods based on modifications for linear codes such as extending, shorting, expurgating, and augmenting. Finally, we summarize and compare the parameters of the discussed constructions.

show abstract

Section: Binary Locally Repairable Codesmentioning

confidence: 99%

Overview of Binary Locally Repairable Codes for Distributed Storage Systems

Kim

2019

Electronics

View full text Add to dashboard Cite

show abstract

“…In [9], [10], [11], [12] binary LRCs are proposed. In [13], a bound on k is obtained in terms of n, k, and r as well as finite field order q.…”

Section: B Related Workmentioning

confidence: 99%

“…It is notable that linear block codes currently in use-e.g. the (16,10,5) code used in Facebook HDFS-RAID, the (16,12,4) code used in Windows Azure Storage, and the (9,6,4)…”

Section: B a Tight Lower Bound On R Of Lrcs Withmentioning

confidence: 99%

On the Average Locality of Locally Repairable Codes

Shahabinejad

Khabbazian

Ardakani

2018

IEEE Trans. Commun.

Self Cite

View full text Add to dashboard Cite

A linear block code with dimension k, length n, and minimum distance d is called a locally repairable code (LRC) with locality r if it can retrieve any coded symbol by at most r other coded symbols. LRCs have been recently proposed and used in practice in distributed storage systems (DSSs) such as Windows Azure storage and Facebook HDFS-RAID. Theoretical bounds on the maximum locality of LRCs (r) have been established. The average locality of an LRC (r) directly affects the costly repair bandwidth, disk I/O, and number of nodes involved in the repair process of a missing data block. There is a gap in the literature studying r. In this paper, we establish a lower bound on r of arbitrary (n, k, d) LRCs. Furthermore, we obtain a tight lower bound on r for a practical case where the code rate (R = k n ) is greater than (1 − 1 √ n ) 2 . Finally, we design three classes of LRCs that achieve the obtained bounds on r. Comparing with the existing LRCs, our proposed codes improve the average locality without sacrificing such crucial parameters as the code rate or minimum distance.

show abstract

“…It meets the need of streaming data availability and super file dealing due to its features of high fault-tolerant capability, high reliability and high throughput rate [15] . The way of data storage and collection in the Hadoop platform can directly influence the availability of the electric data, which is mainly accessed by the data durability in the network.…”

Section: Data Storagementioning

confidence: 99%

Managing and Analyzing Big Error Data for Intelligent Electric Energy Meter Quality Monitoring

Yu¹,

Wang²,

Wei³

2016

dteees

View full text Add to dashboard Cite

Abstract.With the installation and application of intelligent electric energy meters, the need for the veracity of meters from the users is increased thereupon. How to monitor and predict the quality of intelligent electric energy meters based on big error data acquired from intelligent electric energy meters is non-trivial. However, managing and analyzing large volumes of intelligent electric energy meter error data is a great challenge and is consequently hindering the effective utilization of the big dataset collected. In this paper, techniques for collecting, storing and analyzing large volumes of error data is presented to reduce the uncertainty of intelligent electric energy meters quality estimation. First, the error data of intelligent electric energy meters is sampled automatically and accurately by a designed device called Self-Service Error Calibrator Device. Then a storage system based on fountain codes is designed to store the error data on the HDFS platform. At last, an improved data mining technology named HMM (Hidden Markov Model) is leveraged to predict the variation performance tendency of intelligent electric energy meters varied with power load, manufacturers. Simulation results show that our proposed system can provide efficient, accurate, and cost-effective big error data management for intelligent electric energy meters quality monitoring.

show abstract

An Efficient Binary Locally Repairable Code for Hadoop Distributed File System

Cited by 48 publications

References 7 publications

Overview of Binary Locally Repairable Codes for Distributed Storage Systems

Overview of Binary Locally Repairable Codes for Distributed Storage Systems

On the Average Locality of Locally Repairable Codes

Managing and Analyzing Big Error Data for Intelligent Electric Energy Meter Quality Monitoring

Contact Info

Product

Resources

About