Fast and Memory-Efficient Discovery of the Top-k Relevant Subgroups in a Reduced Candidate Space

Abstract

We consider a modified version of the top-k subgroup discovery task, where subgroups dominated by other subgroups are discarded. The advantage of this modified task, known as relevant subgroup discovery, is that it avoids redundancy in the outcome. Although it has been applied in many applications, so far no efficient exact algorithm for this task has been proposed. Most existing solutions do not guarantee the exact solution (as a result of the use of non-admissible heuristics), while the only exact solution relies on the explicit storage of the whole search space, which results in prohibitively large memory requirements.In this paper, we present a new top-k relevant subgroup discovery algorithm which overcomes these shortcomings. Our solution is based on the fact that if an iterative deepening approach is applied, the relevance check -which is the root of the problems of all other approaches -can be realized based solely on the best k subgroups visited so far. The approach also allows for the integration of admissible pruning techniques like optimistic estimate pruning. The result is a fast, memory-efficient algorithm which clearly outperforms existing top-k relevant subgroup discovery approaches. Moreover, we analytically and empirically show that it is competitive with simpler approaches which do not consider the relevance criterion.

An enhanced relevance criterion for more concise supervised pattern discovery

“…Even if such a threshold is not specified beforehand, the threshold can dynamically be determined using the quality of the best k relevant subgroups so far considered. The overall algorithm is quite similar to the one in [8], and the computation is thus roughly as expensive as classical relevant pattern mining. The details are omitted for space reasons, computational aspects not being the focus of our investigation.…”

Flexibly Mining Better Subgroups

“…Traditionally, subgroup discovery focuses on nominal attributes [4,6,7,9]. More recent work [2,11,13,20] considers numeric attributes, employing equal-width or equalfrequency binning to create binary features.…”

Diverse subgroup set discovery

“…More recently, more theoretical work has emerged (Garriga et al 2008;Grosskreutz et al 2008;Grosskreutz and Paurat 2011;Lemmerich et al 2010), in part inspired by more theoretical work in the unsupervised counterpart of SD, frequent pattern mining. Much of this work focuses on the exhaustive, yet efficient traversal of the subgroup hypothesis space, while pruning parts of the search space that can be proven not to yield candidate subgroups of high quality.…”