ComMapReduce: An improvement of MapReduce with lightweight communication mechanisms

Ding, Linlin; Wang, Guoren; Xin, Junchang; Wang, Xiaoyang; Huang, Shongming; Zhang, Rui

doi:10.1016/j.datak.2013.04.004

Cited by 21 publications

(12 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are several computing frameworks, e.g., Hadoop [56], SHadoop [57], ComMapReduce [58], Dryad [59], Piccolo [60], and IBM parallel machine learning toolbox, such systems have the capabilities to scale up machine learning. The combination of deep learning and parallel training implementation techniques provides potential ways to process Big Data [61].…”

Section: Big Data Computingmentioning

confidence: 99%

Efficient Machine Learning for Big Data: A Review

et al. 2015

View full text Add to dashboard Cite

With the emerging technologies and all associated devices, it is predicted that massive amount of data will be created in the next few yearsin fact, as much as 90% of current data were created in the last couple of yearsa trend that will continue for the foreseeable future. Sustainable computing studies the process by which computer engineer/scientist designs computers and associated subsystems efficiently and effectively with minimal impact on the environment. However, current intelligent machine-learning systems are performance driventhe focus is on the predictive/classification accuracy, based on known properties learned from the training samples. For instance, most machine-learning-based nonparametric models are known to require high computational cost in order to find the global optima. With the learning task in a large dataset, the number of hidden nodes within the network will therefore increase significantly, which eventually leads to an exponential rise in computational complexity. This paper thus reviews the theoretical and experimental data-modeling literature, in large-scale data-intensive fields, relating to: (1) model efficiency, including computational requirements in learning, and data-intensive areas' structure and design, and introduces (2) new algorithmic approaches with the least memory requirements and processing to minimize computational cost, while maintaining/improving its predictive/classification accuracy and stability.

show abstract

Section: Big Data Computingmentioning

confidence: 99%

Efficient Machine Learning for Big Data: A Review

et al. 2015

View full text Add to dashboard Cite

show abstract

“…The success of MapReduce stems from hiding the details of parallelization, fault-tolerance, and load balancing in a simple programming framework. MapReduce and its variants are used to process the big data applications, such as Web indexing, data mining, machine learning, financial analysis, scientific simulation, and bioinformatics research [2][3][4][5][6][7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Efficient ELM-Based Two Stages Query Processing Optimization for Big Data

Ding

Liu

Song

et al. 2015

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

MapReduce and its variants have emerged as viable competitors for big data analysis with a commodity cluster of machines. As an extension of MapReduce, ComMapReduce realizes the lightweight communication mechanisms to enhance the performance of query processing applications for big data. However, different communication strategies of ComMapReduce can substantially affect the executions of query processing applications. Although there is already the research work that can identify the communication strategies of ComMapReduce according to the characteristics of the query processing applications, some drawbacks still exist, such as relative simple model, too much user participation, and relative simple query processing execution. Therefore, an efficient ELM-based two stages query processing optimization model is proposed in this paper, named ELM to ELM (E2E) model. Then, we develop an efficient sample training strategy to train ourE2Emodel. Furthermore, two query processing executions based on theE2Emodel, respectively, Just-in-Time execution and Queue execution, are presented. Finally, extensive experiments are conducted to verify the effectiveness and efficiency of theE2Emodel.

show abstract

“…There are some enhancements to MapReduce. For example, in the work [5], classic MapReduce was optimized to decrease the data transformation load. A shared area for information was considered in this approach.…”

Section: Introductionmentioning

confidence: 99%

A MapReduce Based Distributed Framework for Similarity Search in Healthcare Big Data Environment

Sarma

Dwivedi

Rana

et al. 2015

Open and Big Data Management and Innovation

View full text Add to dashboard Cite

Similarity search in the big data environment is a challenging task. Patient Similarity search (PaSi) is an important issue in healthcare network and data. The results of PaSi search may be highly useful for drawing different conclusions and decisions to improve healthcare systems. Such findings can also be useful for choosing the treatment paths for new patients. In this paper, we propose a MapReduce based framework as a solution to the PaSi problem in the context of a healthcare network imagined to be implemented considering the healthcare centers of India. It is assumed that such a healthcare network will be implemented in future over the Government of India cloud known as GI cloud or 'MeghRaj'. The paper also discusses the associated implementation challenges of the proposed framework and the query handling approach for the proposed framework to solve the PaSi problem is stated. Finally, the paper outlines the future scope of the work.

show abstract

ComMapReduce: An improvement of MapReduce with lightweight communication mechanisms

Cited by 21 publications

References 38 publications

Efficient Machine Learning for Big Data: A Review

Efficient Machine Learning for Big Data: A Review

Efficient ELM-Based Two Stages Query Processing Optimization for Big Data

A MapReduce Based Distributed Framework for Similarity Search in Healthcare Big Data Environment

Contact Info

Product

Resources

About