An improved parallel association rules algorithm based on MapReduce framework for big data

Zhou, Xingchen; Huang, Yongfeng

doi:10.1109/fskd.2014.6980847

Cited by 14 publications

(8 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(2012) and Yang et al . (2010) find the frequent 1-itemsets in Phase 1, the frequent 2-itemsets in Phase 2 and so on. In this case, the pruning by the minimum support is done in each phase.…”

Section: Review Of Apriori Implementations On Hadoop-mapreducementioning

confidence: 99%

“…In Lin et al (2012), the authors describe an approach of k phases, known as DPC (Dynamic Passes Combined-Counting), which is capable of generating candidate itemsets of more than one size per iteration, dynamically. In Zhou and Huang (2014), the authors present an approach, also of k phases, known as CPA (Complete Parallel Apriori), which in addition to the step that counts frequent itemsets, it also parallels the generation of candidate itemsets by running two MapReduce per iteration.…”

Section: Review Of Apriori Implementations On Hadoop-mapreducementioning

confidence: 99%

“…There are several proposals for Apriori algorithm implementations on Hadoop-MapReduce (Yang et al ., 2010; Li and Zhang, 2011; Li et al ., 2012; Lin et al ., 2012; Yahya et al ., 2012; Farzanyar and Cercone, 2013a, 2013b; Zhou & Huang, 2014) and Spark (Qiu et al ., 2014; Rathee et al ., 2015). These algorithms can be classified by the number of MapReduce executions, called phases, which are necessary to produce all the frequent itemsets.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Review and comparison of Apriori algorithm implementations on Hadoop-MapReduce and Spark

Castro

Maia

Pereira

et al. 2018

The Knowledge Engineering Review

View full text Add to dashboard Cite

Several Apriori algorithm implementations for mining association rules have been proposed in the literature using the Hadoop-MapReduce framework and, more recently, Spark. However, none of the works have made a detailed assessment of its performance, for example, comparing it with other implementations in various characteristics of data sets. In this work, we present a review of the main algorithms proposed for Hadoop-MapReduce and compared their implementations in a single environment under several different situations. Moreover, these algorithms had their implementations adapted to Spark, and also compared under the same circumstances. Based on the results of the experiments, we present a framework for recommending the Apriori implementation most appropriate for solving a given problem, according to the data set characteristics and minimum required support. The results show that Spark implementations overcome Hadoop-MapReduce implementations at runtime in most experiments. However, there is no single implementation that is the best in all the evaluated situations.

show abstract

Section: Review Of Apriori Implementations On Hadoop-mapreducementioning

confidence: 99%

Section: Review Of Apriori Implementations On Hadoop-mapreducementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Review and comparison of Apriori algorithm implementations on Hadoop-MapReduce and Spark

Castro

Maia

Pereira

et al. 2018

The Knowledge Engineering Review

View full text Add to dashboard Cite

show abstract

“…Association rules are employed today in many application areas, an area of specific interest being Web usage mining (Agrawal & Srikant 1994;Kriegel 2007). Association rules are usually required to satisfy a user-specified minimum support and a user-specified minimum confidence simultaneously (Kum et al 2005;Hacibeyoglu et al 2013;Ganapathy et al 2014;Zhou & Huang 2014). Association rule generation is usually split up into two separate steps: First, the minimum support being applied to find all frequent itemsets in a database.…”

Section: Association Rule Miningmentioning

confidence: 99%

Prediction of users webpage access behaviour using association rule mining

GeethaRamani

Revathy²,

Jacob³

2015

Sadhana

View full text Add to dashboard Cite

Web Usage mining is a technique used to identify the user needs from the web log. Discovering hidden patterns from the logs is an upcoming research area. Association rules play an important role in many web mining applications to detect interesting patterns. However, it generates enormous rules that cause researchers to spend ample time and expertise to discover the really interesting ones. This paper works on the server logs from the MSNBC dataset for the month of September 1999. This research aims at predicting the probable subsequent page in the usage of web pages listed in this data based on their navigating behaviour by using Apriori prefix tree (PT) algorithm. The generated rules were ranked based on the support, confidence and lift evaluation measures. The final predictions revealed that the interestingness of pages mainly depended on the support and lift measure whereas confidence assumed a uniform value among all the pages. It proved that the system guaranteed 100% confidence with the support of 1.3E−05. It revealed that the pages such as Front page, On-air, News, Sports and BBS attracted more interested subsequent users compared to Travel, MSN-News and MSN-Sports which were of less interest.

show abstract

“…Zhou, X., et al [38] proposed an improved parallel association rules algorithm utilizing Hadoop as the MapReduce distributed programming framework. It has shown that the algorithm achieve well based on parallel performance and could be easily realized with the Hadoop platform.…”

Section: Related Workmentioning

confidence: 99%

A Novel Mapreduce Lift Association Rule Mining Algorithm (Mrlar) for Big Data

Oweis¹,

Fouad²,

Oweis³

et al. 2016

ijacsa

View full text Add to dashboard Cite

Abstract-Big Data mining is an analytic process used to discover the hidden knowledge and patterns from a massive, complex, and multi-dimensional dataset. Single-processor's memory and CPU resources are very limited, which makes the algorithm performance ineffective. Recently, there has been renewed interest in using association rule mining (ARM) in Big Data to uncover relationships between what seems to be unrelated. However, the traditional discovery ARM techniques are unable to handle this huge amount of data. Therefore, there is a vital need to scalable and parallel strategies for ARM based on Big Data approaches. This paper develops a novel MapReduce framework for an association rule algorithm based on Lift interestingness measurement (MRLAR) which can handle massive datasets with a large number of nodes. The experimental result shows the efficiency of the proposed algorithm to measure the correlations between itemsets through integrating the uses of MapReduce and LIM instead of depending on confidence.

show abstract

An improved parallel association rules algorithm based on MapReduce framework for big data

Cited by 14 publications

References 3 publications

Review and comparison of Apriori algorithm implementations on Hadoop-MapReduce and Spark

Review and comparison of Apriori algorithm implementations on Hadoop-MapReduce and Spark

Prediction of users webpage access behaviour using association rule mining

A Novel Mapreduce Lift Association Rule Mining Algorithm (Mrlar) for Big Data

Contact Info

Product

Resources

About