Dynamic itemset counting and implication rules for market basket data

Abstract

We consider the problem of analyzing market-basket data and present several important contributions. First, we present a new algorithm for finding large itemsets which uses fewer passes over the data than classic algorithms, and yet uses fewer candidate itemsets than methods based on sampling. We investigate the idea of item reordering, which can improve the low-level efficiency of the algorithm. Second, we present a new way of generating “implication rules,” which are normalized based on both the antecedent and the consequent and are truly implications (not simply a measure of co-occurrence), and we show how they produce more intuitive results than other methods. Finally, we show how different characteristics of real data, as opposed by synthetic data, can dramatically affect the performance of the system and the form of the results.

Untitled

“…His first work in this direction aimed at finding general and efficient frameworks for mining so-called association rules, popularized by Rakesh Agarwal and his colleagues in the then nascent field of data mining. Together with his colleagues in the Stanford database group, Motwani wrote several influential papers [12,11,23] proposing new frameworks and algorithms for finding association rules and their variants.…”

Multipass Algorithms for Mining Association Rules in Text Databases

“…Traditional domains for finding frequent itemsets, and subsequently the association rules, include retail point-of-sale (POS) transaction databases and catalog order databases (Brin et al, 1997). The natural item instances are the sales transaction items, but other item instances are possible.…”

“…Mining association rules in transaction databases has been demonstrated to be useful and technically feasible in several application areas (Brin et al, 1997;Chen et al, 1996), particularly in retail sales. Let I = {i 1 , i 2 , .…”

Knowledge Discovery in Textual Databases: A Concept-Association Mining Approach

“…Recent works [5] [6] [7] [8] deal with finding rules based on other metrics besides support and confidence. In [6], the authors mine association rules that identify correlations and consider both the absence and presence of items as a basis for generating the rules.…”

“…In [6], the authors mine association rules that identify correlations and consider both the absence and presence of items as a basis for generating the rules. In [7], the authors use support as part of their measure of interest of an association. However, when rules are generated, instead of using confidence, the authors use a metric they call conviction, which is a measure of implication and not just co-occurrence.…”