A fast map-reduce algorithm for burst errors in big data cloud storage

Abstract

In distributed storage for Big Data systems, there is a need for exact repair, high bandwidth codes. The challenge for exact repair in big-data storage is to simultaneously enable both very high bandwidth repair using Map-Reduce and simple coding schemes that also combine robust maximally distance separable (MDS) exact repair. MDS repair is for the rare, but exceptional outlier error patterns requiring optimum erasure code reconstruction. We construct the optimum fast bandwidth repair for big-data sources. Our system uses Map-Reduce, exact repair reconstruction. The algorithm combines MDS with a second fast decode algorithm in a cloud environment. We illustrate cloud experiments for optimum fast bandwidth reconstruction for 1-Exabyte Big Data in the cloud and demonstrate cloud results for Poisson error rate arrival models. Unlike prior methods, we jointly solve the problem of fast bandwidth repair for burst-memory error patterns and for code rates up to in a real time error model framework for Big Data. Furthermore, simulations indicate this method outperforms prior fast bandwidth approaches for burst errors. We also illustrate Map-Reduce algorithm optimized for fast bandwidth repair in Big Data storage in clouds.

Location Estimation of a Photo: A Geo-signature MapReduce Workflow

“…Customarily, the purpose of "MapReduce approach" [51][52][53] was designed for the actual "dynamic structure" for the "velocity", "volume" and "variety" of non-volatile large-scale datasets (or Big data [1]). In case of a geo-untagged photo, MapReduce indexing filters only the useful geo-tagged photos (that were collected in term of vectors with geo-tagging) from the set of large-scale samples which are similar to some visual contents of geo-untagged photo.…”

Robust Fuzzy Neuro System for Big Data Analytics

“…Each node of cluster is broken down into key/value pairs. [4] The different phases included in it are • sorting,…”