i2MapReduce: Incremental mapreduce for mining evolving big data

Zhang, Yanfeng; Chen, Shimin; Wang, Qiang; Yu, Ge

doi:10.1109/icde.2016.7498385

Cited by 17 publications

(37 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hadoop were used by [10] to provide a way of sentiment analysis for processing the huge amount of data on a Hadoop cluster for faster in real time execution. Hadoop was again find its place in [11]. MapReduce were used to perform key-value pair level incremental processing in this work.…”

Section: Literature Reviewmentioning

confidence: 99%

Extracting the Sentiment Score of Customer Review from Unstructured Big Data Using Map Reduce Algorithm

Hassan¹

2016

IJDTA

View full text Add to dashboard Cite

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Extracting the Sentiment Score of Customer Review from Unstructured Big Data Using Map Reduce Algorithm

Hassan¹

2016

IJDTA

View full text Add to dashboard Cite

show abstract

“…iiHadoop performs incremental iterative computation on the level of key-value pairs as presented in [16] Thus, the preserved values from IR for each k2 are retrieved and merged with the corresponding values from IR to obtain the updated input of Reduce tasks. The other k2 in the preserved IR is not changed and therefore would generate the same final result.…”

Section: Basic Ideamentioning

confidence: 99%

“…In this paper, we present a new framework called iiHadoop, that extends both the traditional Hadoop MapReduce and i 2 MapReduce to support efficient incremental iterative computations. iiHadoop overcomes the limitations of existing systems and efficiently implements incremental computation using key-value pair level approach presented in i 2 MapReduce [16]. iiHadoop presents the following main contributions:…”

mentioning

confidence: 99%

See 1 more Smart Citation

iiHadoop: an asynchronous distributed framework for incremental iterative computations

Saadon

Mokhtar

2017

J Big Data

View full text Add to dashboard Cite

IntroductionToday, a large amount of data is being produced in many areas including: e-commerce, social network, finance, health-care, and education. This increase in data volume consequently increases the need for an efficient computing framework to process this data and transform it into meaningful information. In the past years, many distributed computing frameworks [1][2][3][4][5][6] have been developed to perform large-scale data processing. MapReduce [2] (with its open-source implementation, Hadoop [7]) is the most widely used framework because of its simplicity, scalability, efficiency, and reliability. It was proposed by Google in 2004 to enable the distribution and processing of large-scale data over a cluster of commodity machines. The framework takes care of the distributed execution, fault tolerance, load balancing, and job scheduling; therefore, developers and programmers do not need to worry about these issues and can simply perform their tasks. AbstractIt is true that data is never static; it keeps growing and changing over time. New data is added and old data can either be modified or deleted. This incremental nature of data motivates the development of new systems to perform large-scale data computations incrementally. MapReduce was recently introduced to provide an efficient approach for handling large-scale data computations. Nevertheless, it turned to be inefficient in supporting the processing of small incremental data. While many previous systems have extended MapReduce to perform iterative or incremental computations, these systems are still inefficient and too expensive to perform large-scale iterative computations on changing data. In this paper, we present a new system called iiHadoop, an extension of Hadoop framework, optimized for incremental iterative computations. iiHadoop accelerates program execution by performing the incremental computations on the small fraction of data that is affected by changes rather than the whole data. In addition, iiHadoop improves the performance by executing iterations asynchronously, and employing locality-aware scheduling for the map and reduce tasks taking into account the incremental and iterative behavior. An evaluation for the proposed iiHadoop framework is presented using examples of iterative algorithms, and the results showed significant performance improvements over comparable existing frameworks.

show abstract

“…Hence it becomes mandatory to model a system which is capable of handling of the data evolution in the large data. Incoop [2] and I2map reduce [3] have used for evolution data processing by extending MapReduce tosupport incremental processing. However, it has two mainlimitations.…”

Section: Introductionmentioning

confidence: 99%

Evolution Mining: A Novel Accelerated Framework for Large Data processing in the big Data Paradigm

Rajalakshmi¹

2017

IJRASET

View full text Add to dashboard Cite

The big data focus on the mining of the large volume, complex and exploiting datain the large scale application and Social network communities. The map reduce paradigm is been used to handle large scale data in the big data paradigm through classification and clustering algorithm as it is treated as computational intensive task. The Data Classification and Clustering algorithm using supervised and unsupervised learning models failed in processing dynamic updates as concept and feature evolution to map reduce paradigm in the large streaming application. Therefore it becomes mandatory to build a new accelerated framework which is capable of handling of the data evolution in the large data. In this Paper, we propose a novel technique named as "EvolutionMining" which develop a deep learning model in the map reduce paradigm using key value pair processing &Semantic Mining techniques. Sophisticated iterative models for feature extraction, classification and feature or concept evolution determination been devised. The proposed Technique is capable of handling the diverse data by producing the increased value to the computation characteristics such as accuracy and complexity. The Experimental results prove that proposed System outperforms other state of approaches using evaluation metrics such Runtime &Mean Error and reduces I/O overhead to much extent.

show abstract

i2MapReduce: Incremental mapreduce for mining evolving big data

Cited by 17 publications

References 9 publications

Extracting the Sentiment Score of Customer Review from Unstructured Big Data Using Map Reduce Algorithm

Extracting the Sentiment Score of Customer Review from Unstructured Big Data Using Map Reduce Algorithm

iiHadoop: an asynchronous distributed framework for incremental iterative computations

Evolution Mining: A Novel Accelerated Framework for Large Data processing in the big Data Paradigm

Contact Info

Product

Resources

About