Model-based probabilistic frequent itemset mining

Abstract

Data uncertainty is inherent in emerging applications such as location-based services, sensor monitoring systems, and data integration. To handle a large amount of imprecise information, uncertain databases have been recently developed. In this paper, we study how to efficiently discover frequent itemsets from large uncertain databases, interpreted under the Possible World Semantics. This is technically challenging, since an uncertain database induces an exponential number of possible worlds. To tackle this problem, we propose a novel methods to capture the itemset mining process as a probability distribution function taking two models into account: the Poisson distribution and the normal distribution. These model-based approaches extract frequent itemsets with a high degree of accuracy and support large databases. We apply our techniques to improve the performance of the algorithms for (1) finding itemsets whose frequentness probabilities are larger than some threshold and (2) mining itemsets with the k highest frequentness probabilities. Our approaches support both tuple and attribute uncertainty models, which are commonly used to represent uncertain databases. Extensive evaluation on real and synthetic datasets shows that our methods are highly accurate and four orders of magnitudes faster than previous approaches. In further theoretical and experimental studies, we give an intuition which model-based approach fits best to different types of data sets.

A regression-based algorithm for frequent itemsets mining

“…From another view, studies involving web-based semantic itemset mining (Lazcorreta et al , 2008), utility-based itemset mining methods (Yao and Hamilton, 2006), pre-pruned tree-based itemset mining (Meng and Sha, 2017), and possible world semantic mining (Bernecker et al , 2013) focused on mining rules by setting semantics on items. These semantic-based algorithms set semantic definitions of associations to enhance the interpretation and application of mined items.…”

Data mining for decision support with uncertainty on the airplane

“…For Itemset mining in uncertain data [39,40], two main models are generally used, (1) the expected support model [41] (an itemset X is considered frequent if and only if its expected support is not less than a user-specified support threshold) and (2) the probabilistic frequentness model [42] (an itemset X is called frequent if the probability that X occurs in at least minSup transactions is above a given threshold). In the first approach, the basic idea consists in exploiting the statistical properties of those items with low existential probabilities with a framework that comprises three modules: the trimming module, pruning module and patch up module.…”

Probabilistic Frequent Itemset Mining with Hierarchical Background Knowledge

“…Considering higher levels allows us to mine rules which would not be learned ( ⊆ )). Bernecker et al [11] performed a thorough otherwise, and to learn more concise and generalized rules. The authors propose methods which explore taxonomies to speed up the mining process.…”