Multiagent-consensus-MapReduce-based attribute reduction using co-evolutionary quantum PSO for big data applications

Ding, Weiping; Lin, Chin‐Teng; Chen, Senbo; Zhang, Xiaofeng; Hu, Bin

doi:10.1016/j.neucom.2017.06.059

Cited by 38 publications

(14 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the efficiency of existing MCM-AR [4], FW-AR [5] and SVD based Attribute Reduction (SVD-AR) […”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Parallel Rough Set Theory for Nonlinear Dimension-Reduction in Big Data Analysis

Muthusamy¹,

Subramani²

2019

IJIES

View full text Add to dashboard Cite

Attribute reduction for big data is an important preprocessing step in the area of data mining. A multi-step dimension reduction approach was proposed for attribute reduction in big data. It addressed the non-linear relationships within the attributes. The data dimension was reduced through a parametric mapping. The mapping parameters were estimated using low-rank Singular Value Decomposition (SVD). However, the user-defined criterion in multi-step dimension reduction approach has greatly influenced the efficiency of attribute reduction. This approach was proposed for a single machine that means the entire big data must fit in the main memory and the parallelism was limited. So, in this paper, parallel rough set theory based attribute reduction approach is proposed for attribute reduction in big data. Based on two descriptions of lower approximation and upper approximation, a rough set is constructed. Then a reduct is detected using inner importance measure and outer importance measure. The rough set theory is used in MapReduce framework to achieve the parallelism for attribute reduction in big data. Hence, the computation time is reduced by using parallel rough set theory based attribute reduction approach. Finally, the experiments are carried out in Amazon customer review, REUTERS-21578 and International Cancer Genome Consortium (ICGC) on AWS datasets to prove the effectiveness of parallel rough set theory based attribute reduction in terms of accuracy, precision, recall and computation time.

show abstract

“…In this section, the efficiency of existing MCM-AR [4], FW-AR [5] and SVD based Attribute Reduction (SVD-AR) […”

Section: Resultsmentioning

confidence: 99%

“…For attribute reduction in big data, a Multiagent-Consensus-MapReduce-based Attribute Reduction (MCMAR) algorithm [4] was proposed. A Particle Swarm Optimization (PSO) with self-adaptive memeplexes was designed to partition the particles into different memeplexes which located global best region.…”

Section: Literature Surveymentioning

confidence: 99%

A Parallel Rough Set Theory for Nonlinear Dimension-Reduction in Big Data Analysis

Muthusamy¹,

Subramani²

2019

IJIES

View full text Add to dashboard Cite

show abstract

“…An attribute reduction algorithm was offered to systematically advance the attribute reduction efficacy based on the Multi-agent Consensus MapReduce (MCMAR) optimization model and co-evolutionary quantum Particle Swarm Optimization (PSO) with self-adaptive memeplexes. The results show that MCMAR has better feasibility and efficiency than previous algorithms, which can clearly enhance the superior performance of attribute reduction in big datasets (Ding et al, 2018).…”

Section: Methodsmentioning

confidence: 94%

“…Attribute or variable reduction through feature selection is a central application in RST. However, because there are often many redundant or non-redundant, relevant or irrelevant features in practical problems, it is not easy to extract features due to high interaction and interdependency (Ding et al, 2018).…”

Section: Methodsmentioning

confidence: 99%

Healthcare Driven by Big Data Analytics

Alexander

Wang²

2018

American Journal of Engineering and Applied Sciences

View full text Add to dashboard Cite

Text messages and social network posts are often included in big data and are frequently invaluable sources of health data. When machine learning or data mining is used, it is important to perform an automatic process for integrating all available and related health data. Artificial Neural Network (ANN) is one of the machine learning methods flexible in using algorithms to detect complicated nonlinear relationships within huge datasets. Pharmacokinetics uses genetics to individualize drug therapy. Because genetic and pharmacological analysis often require large scale computation, Big Data analytics has shown potential in this area. Personal data from various sensors can often be used for making health and treatment recommendations, taking appropriate action for a patient's lifestyle choices and early diagnosis vital to advancing quality of care. Big data techniques such as hadoop and spark have been used in various areas of healthcare and big data analytics has been employed in great datasets to expose hidden patterns or correlations for effective decision-making. Challenges of big data in healthcare, concerned with gathering material from multifaceted heterogonous patient sources still exist, although hadoop has been employed as the processing unit for heterogeneous data gathered from various body sensors. The most critical necessity in a healthcare big data system is the data security; however, users are continuously applying big data-driven strategies to solve the various problems and challenges.

show abstract

“…The performance of this approach was compared with the state-ofthe-art techniques and resulted in better performance. An attribute reduction method based on a multiagent-consensus MapReduce model for big data applications has been proposed using a co-evolutionary quantum PSO with self-adaptive memeplexes to group the particles into different memeplexes (Ding et al, 2018). A four-layer neighborhood radius framework with a compensatory scheme splits the attribute sets into subspace maintaining attributes interacting properties and maps to the MapReduce model.…”

Section: Co-evolution and Mapreduce-based Feature Selection Approachesmentioning

confidence: 99%

Knowledge management overview of feature selection problem in high-dimensional financial data: cooperative co-evolution and MapReduce perspectives

Rashid

Choudhury

2019

Problems and Perspectives in Management

View full text Add to dashboard Cite

The term “big data” characterizes the massive amounts of data generation by the advanced technologies in different domains using 4Vs – volume, velocity, variety, and veracity - to indicate the amount of data that can only be processed via computationally intensive analysis, the speed of their creation, the different types of data, and their accuracy. High-dimensional financial data, such as time-series and space-time data, contain a large number of features (variables) while having a small number of samples, which are used to measure various real-time business situations for financial organizations. Such datasets are normally noisy, and complex correlations may exist between their features, and many domains, including financial, lack the al analytic tools to mine the data for knowledge discovery because of the high-dimensionality. Feature selection is an optimization problem to find a minimal subset of relevant features that maximizes the classification accuracy and reduces the computations. Traditional statistical-based feature selection approaches are not adequate to deal with the curse of dimensionality associated with big data. Cooperative co-evolution, a meta-heuristic algorithm and a divide-and-conquer approach, decomposes high-dimensional problems into smaller sub-problems. Further, MapReduce, a programming model, offers a ready-to-use distributed, scalable, and fault-tolerant infrastructure for parallelizing the developed algorithm. This article presents a knowledge management overview of evolutionary feature selection approaches, state-of-the-art cooperative co-evolution and MapReduce-based feature selection techniques, and future research directions.

show abstract

Multiagent-consensus-MapReduce-based attribute reduction using co-evolutionary quantum PSO for big data applications

Cited by 38 publications

References 36 publications

A Parallel Rough Set Theory for Nonlinear Dimension-Reduction in Big Data Analysis

A Parallel Rough Set Theory for Nonlinear Dimension-Reduction in Big Data Analysis

Healthcare Driven by Big Data Analytics

Knowledge management overview of feature selection problem in high-dimensional financial data: cooperative co-evolution and MapReduce perspectives

Contact Info

Product

Resources

About