Mining Low-Support Discriminative Patterns from Dense and High-Dimensional Data

Abstract

Discriminative patterns can provide valuable insights into datasets with class labels, that may not be available from the individual features or the predictive models built using them. Most existing approaches work efficiently for sparse or low-dimensional datasets. However, for dense and highdimensional datasets, they have to use high thresholds to produce the complete results within limited time, and thus, may miss interesting low-support patterns. In this paper, we address the necessity of trading off the completeness of discriminative pattern discovery with the efficient discovery of lowsupport discriminative patterns from such datasets. We propose a family of anti-monotonic measures named SupMaxK that organize the set of discriminative patterns into nested layers of subsets, which are progressively more complete in their coverage, but require increasingly more computation. In particular, the member of SupMaxK with K = 2, named SupMaxPair , is suitable for dense and high-dimensional datasets. Experiments on both synthetic datasets and a cancer gene expression dataset demonstrate that there are low-support patterns that can be discovered using SupMaxPair but not by existing approaches. Furthermore, we show that the low-support discriminative patterns that are only discovered using SupMaxPair from the cancer gene expression dataset are statistically significant and biologically relevant. This illustrates the complementarity of SupMaxPair to existing approaches for discriminative pattern discovery. The codes and dataset for this paper are available at http://vk.cs.umn.edu/SMP/.

Applying data mining techniques to address critical process optimization needs in advanced manufacturing

“…Specifically, it helps discover parameter recipes as well as sets of feature values which are closely related to qualified panels and defective panels. In PDP-Miner, the techniques of association based classification [16] and low-support discriminative pattern mining [6] are leveraged to conduct the discriminative analysis.…”

“…To address this problem, we adapt the idea of low support pattern mining algorithm (SMP ) [6] and integrate the algorithm into PDP-miner. SMP aims at mining the discriminative patterns by leveraging a family of anti-monotonic measures called SupMaxK.…”

“…A naïve implementation using this measure suffers from low efficiency [6] when pruning frequent non-discriminative rules. To address this issue, a new measure -SupM axK(α) is introduced to help prune unrelated patterns by estimating S def ective (β).…”

RP-growth: Top-k Mining of Relevant Patterns with Minimum Support Raising

“…The difference from our method is that we do not perform the merging operation introduced in [6]. 7 This example was originally introduced in [20].…”

Mining emerging patterns by streaming feature selection

“…In addition, a Jump Emerging Pattern (JEP for short) is a special type of EPs whose support increases from zero in one class to non-zero in the other class [12]. Like other patterns composed of conjunctive combinations of elements, EPs can be easily understood and used directly in a wide range of applications, such as failure detection [16] and discovering knowledge in gene expression data [8,18].…”