Scaling associative classification for very large datasets

Abstract

IntroductionIn the recent years, Big Data have received much attention by both the academic and the industrial world, with the aim of fully leveraging the power of the information they hide. The dimensions on which very large datasets usually extend are mainly the size, i.e. the disk storage occupied, the volume, i.e. the number of records, the dimensionality, i.e. the number of features a record can have, and the domain, i.e. the number of distinct values a feature can take. A special effort has been dedicated to Machine learning algorithms, with a profusion of solutions to tackle the scalability problem, on some or all of the dimensions mentioned above.Scalability on the domain dimension is a special concern for the datasets in which most of the features are categorical. Categorical features have their values expressed in a discrete domain, and no concept of ordering or ranking can be assumed. Discrete or discretized features are a special case of categorical features where an order among the values is defined. The absence of a natural ordering increases the complexity of the treatment of categorical variables, as their values cannot be binned in groups or levels for example. AbstractSupervised learning algorithms are nowadays successfully scaling up to datasets that are very large in volume, leveraging the potential of in-memory cluster-computing Big Data frameworks. Still, massive datasets with a number of large-domain categorical features are a difficult challenge for any classifier. Most off-the-shelf solutions cannot cope with this problem. In this work we introduce DAC, a Distributed Associative Classifier. DAC exploits ensemble learning to distribute the training of an associative classifier among parallel workers and improve the final quality of the model. Furthermore, it adopts several novel techniques to reach high scalability without sacrificing quality, among which a preventive pruning of classification rules in the extraction phase based on Gini impurity. We ran experiments on Apache Spark, on a real large-scale dataset with more than 4 billion records and 800 million distinct categories. The results showed that DAC improves on a state-of-the-art solution in both prediction quality and execution time. Since the generated model is human-readable, it can not only classify new records, but also allow understanding both the logic behind the prediction and the properties of the model, becoming a useful aid for decision makers. Venturini et al. J Big Data (2017) Page 2 of 24 Venturini et al. J Big Data (2017) 4:44 Associative classifiers are a special category of Machine learning algorithms, where association rule mining is exploited for the purpose of classification. In the past, they have proved to be able to produce classification models of high quality and outperform state-of-art algorithms like decision trees [1]. Moreover, the model produced is readable, as it is made of association rules, can be debugged and even manually tuned if needed, by modifying or deleting specific rules. In a world ...

Semi-supervised associative classification using ant colony optimization algorithm

“…where Supp(X =>Y) denotes the support of pattern <if X then Y > while P(X ∪ Y) represents the probability of occurrence of itemset X with class label Y (Hadi, Al-Radaideh & Alhawari, 2018;Nguyen et al, 2018). Definition 7 Confidence of a pattern (X => Y) (Venturini, Baralis & Garza, 2018;Hadi, Al-Radaideh & Alhawari, 2018) is calculated as:…”

Evaluating associative classification algorithms for Big Data

“…• DAC [32]. Ensemble learning which distributes the training of an associative classifier among parallel workers.…”

Associative Classification using Automata with Structure based Merging

“…DM contains a rich set of classification models; specifically, Support Vector Machine [9], Rule Based [10], Decision Tree [11,12], Bayesian classification [2], k -Nearest Neighbor [13], and AC [14]. Among all, AC is relatively new and promising [15,16,17,18,19,20,21,22,23,24] as it combines the best approaches of association rules mining (ARM) and classification. AC is based on ARM where, first, the strongest Class Association Rules (CAR) are discovered from dataset, followed by converting those rules into classifier model.…”

“…After the introduction of AC in 1997, numbers of algorithms are developed in this family e.g. CBA [14,25], CMAR [18], CPAR [21], MCAR [19], MAC [26], CMARAA [27], MRAC & MRAC+ [15], DAC [23], CBA-Spark and CPAR-Spark [24] and G3P-ACBD [28]. Almost all consist of three basic steps -a) Association rule generation, b) Classifier building -rule pruning and rule ranking c) Classification of unknown records using the classifier.…”