D-GENE: Deferring the GENEration of Power Sets for Discovering Frequent Itemsets in Sparse Big Data

Abstract

Sparseness is the distinctive aspect of big data generated by numerous applications at present. Furthermore, several similar records exist in real-world sparse datasets. Based on Iterative Trimmed Transaction Lattice (ITTL), the recently proposed TRICE algorithm learns frequent itemsets efficiently from sparse datasets. TRICE stores alike transactions once, and eliminates the infrequent part of each distinct transaction afterward. However, removing the infrequent part of two or more distinct transactions may result in similar trimmed transactions. TRICE repeatedly generates ITTLs of similar trimmed transactions that induce redundant computations and eventually, affects the runtime efficiency. This paper presents D-GENE, a technique that optimizes TRICE by introducing a deferred ITTL generation mechanism. D-GENE suspends the process of ITTL generation till the completion of transaction pruning phase. The deferral strategy enables D-GENE to generate ITTLs of similar trimmed transactions once. Experimental results show that by avoiding the redundant computations, D-GENE gets better runtime efficiency. D-GENE beats TRICE, FP-growth, and optimized versions of SaM and RElim algorithms comprehensively, especially when the difference between distinct transactions and distinct trimmed transactions is high.INDEX TERMS Big data applications, pattern recognition, association rules, frequent item set mining, IoT.

Association rule mining based fuzzy manta ray foraging optimization algorithm for frequent itemset generation from social media

“…But the execution operation seems bit complex was the major drawback of this approach. Yasir et al 33 demonstrated the Deferring GENEration of Power sets (D‐GENE) that discovers frequent itemsets in sparse big data. Sparseness value, running time, number of transactions, minimum support were the evaluation metrics employed in this approach.…”

D-GENE-Based Discovery of Frequent Occupational Diseases among Female Home-Based Workers

“…Recently, a new FIM algorithm, Deferring the Generation of Power sets for Mining Frequent Itemsets in Sparse Big data (D-GENE), was proposed [12]. D-GENE uses the concept of power set from set theory to generate an Iterative Trimmed Transaction Lattice (ITTL) of each transaction.…”

“…This section covers the working principle and the implementation details of the D-GENE algorithm for mining frequent co-occurring diseases [12]. In the first place, the detailed explanation of the dataset, preprocessing, and transformation steps is provided.…”

“…D-GENE was chosen due to its superior runtime efficiency as well as having the least memory consumption, especially for sparse data. D-GENE has shown its superiority over other state-of-the-art methods like Apriori and FP-growth [12]. Considering the sparse nature of the data used in this study, D-GENE was believed to be the optimal choice to discover frequent co-occurring diseases from them.…”

A Fast Approach for Up-Scaling Frequent Itemsets

“…EAFIM [20] that uses the Apache Spark framework to achieve parallelism is an improved version of the apriori algorithm. Yasir, Muhammad, et al propose the HARPP [21], which adopt the concern of pow set and dictionary data structures, and the D-GENE [22], which suspends the process of ITTL generation till the completion of transaction pruning phase, discovering frequent itemsets from sparse datasets.…”