Re-examination of interestingness measures in pattern mining: a unified framework

Abstract

Numerous interestingness measures have been proposed in statistics and data mining to assess object relationships. This is especially important in recent studies of association or correlation pattern mining. However, it is still not clear whether there is any intrinsic relationship among many proposed measures, and which one is truly effective at gauging object relationships in large data sets. Recent studies have identified a critical property, null-(transaction) invariance, for measuring associations among events in large data sets, but many measures do not have this property. In this study, we re-examine a set of null-invariant interestingness measures and find that they can be expressed as the generalized mathematical mean, leading to a total ordering of them. Such a unified framework provides insights into the underlying philosophy Responsible editor: M.J. Zaki. 123 372 T. Wu et al.of the measures and helps us understand and select the proper measure for different applications. Moreover, we propose a new measure called Imbalance Ratio to gauge the degree of skewness of a data set. We also discuss the efficient computation of interesting patterns of different null-invariant interestingness measures by proposing an algorithm, GAMiner, which complements previous studies. Experimental evaluation verifies the effectiveness of the unified framework and shows that GAMiner speeds up the state-of-the-art algorithm by an order of magnitude.

Efficient Mining of Top Correlated Patterns Based on Null-Invariant Measures

“…This seems to be problem-specific and cannot be captured by a single correlation measure which is the best for all cases. As a result, a number of correlation measures has been proposed [8,16,17,19]. In this work we limit ourselves to null (transaction)-invariant [8,16,17,19] correlation measures based on conditional probabilities.…”

“…As a result, a number of correlation measures has been proposed [8,16,17,19]. In this work we limit ourselves to null (transaction)-invariant [8,16,17,19] correlation measures based on conditional probabilities. They quantify the degree of mutual relationships between items in a group without taking into account the items outside the group in question.…”

“…The importance of null-invariance for uncovering meaningful relationships between objects was analyzed in [19]. If we use correlation measures which are not null-invariant, the relationships between objects may appear or disappear simply by changing the number of transactions which do not contain items in question.…”

“…The five known null-invariant correlation measures are All Confidence, Coherence, Cosine, Kulczynski and Max Confidence [19]. The degree of the correlation is represented as a real number between 0 and 1.…”

“…Out of five measures, only All Confidence and Coherence are anti-monotonic. Unfortunately, using only All Confidence or Coherence may not be appropriate for cases involving unbalanced supports, which was demonstrated in [19]. Strong correlations for such unbalanced cases can be captured if we evaluate the relationships as an average of conditional probabilities.…”