Outlier Ensembles

Ensembles for unsupervised outlier detection

“…The greedy combination strategy [63] also raises an interesting challenge for outlier ensembles: how to choose good ensemble members or how to train improved ensemble members based on previously learned and evaluated models in the absence of a ground truth for evaluation. This challenge has also been noted by Aggarwal [2]. The heuristic of the greedy ensemble will be discussed in Section 4.3.…”

“…Combining models learned using different parameters has been proposed as an ensemble method by Gao and Tan [23], although Aggarwal [2] identified traces of this idea in earlier papers without an explicit discussion of the ensemble idea. For example, for LOF [9] as well as for LOCI [56], the authors suggested to try different parameters (controlling the granularity of neighborhood analysis; in LOCI, the combination of different granularity is even inherent to the method) and to use the granularity that optimally enhances the outlierness.…”

“…One might think of it as a majority vote of a jury (as in Condorcet's Jury theorem [47]): One or another judgement about an observation might be wrong, but the majority might still be right, as long as the judgements are, overall, somewhat reliable and every member decides independently from the others. Aggarwal [2] recently proposed a categorization of ensemble approaches to outlier detection by algorithmic patterns or strategies. He distinguishes "sequential ensembles" vs. "independent ensembles", and "model-centered ensembles" vs. "data-centered ensembles".…”

“…Likewise, it would seem that ensemble techniques such as feature bagging [45], i.e., using different subspaces as a means to learn diverse models, could also benefit from insights in the area of subspace outlier detection. Complementary to Aggarwal [2], we would like to discuss here the specific challenges, the first steps taken so far in the literature, and overall the important questions in research regarding ensembles for outlier detection. Transferring basic principles from supervised learning, the two key principles of ensemble construction would be accuracy and diversity.…”

“…Casting outlier detection as an unsupervised problem, however, there is nothing known about the accuracy of individual outlier detectors during learning. This is a very fundamental problem and, as Aggarwal [2] pointed out, probably one of the main reasons why the state of the art in research on ensembles for unsupervised outlier detection is not very advanced. But obviously this problem would also affect ensemble clustering where we have a lot more of research presented in the literature.…”