Local Outlier Detection with Interpretation

Abstract

Outlier detection aims at searching for a small set of objects that are inconsistent or considerably deviating from other objects in a dataset. Existing research focuses on outlier identification while omitting the equally important problem of outlier interpretation. This paper presents a novel method named LODI to address both problems at the same time. In LODI, we develop an approach that explores the quadratic entropy to adaptively select a set of neighboring instances, and a learning method to seek an optimal subspace in which an outlier is maximally separated from its neighbors. We show that this learning task can be solved via the matrix eigen-decomposition and its solution contains essential information to reveal features that are most important to interpret the exceptional properties of outliers. We demonstrate the appealing performance of LODI via a number of synthetic and real world datasets and compare its outlier detection rates against state-of-the-art algorithms.

Scalable Outlying-Inlying Aspects Discovery via Feature Ranking

“…The latest work on outlying aspects mining can be categorized into two main directions, which we refer to as feature selection approaches [4], and score-andsearch approaches [5].…”

“…More specifically, the two classes are defined as the query point (positive class) and the rest of the data (negative class). In [4], to balance the classes, the positive class is over-sampled with samples drawn from a Gaussian distribution centred at the query point, while the negative class is under-sampled, keeping k full-space neighbors of the query point and some other data points from the rest of the data. Similarly in [12], the positive class is over-sampled while keeping all other data points as the negative class.…”

“…In the feature selection approaches [4,12], to balance the class distribution, the positive class c 1 is augmented with synthetic samples from a Gaussian distribution centred at the query. The task is then to select the top features that distinguish the two classes.…”

Loda: Lightweight on-line detector of anomalies

“…Algorithms presented therein are data-centered and suitable only for low-dimensions. HiCS algorithm (Keller et al 2012) and the algorithm of Dang et al (2013) both aim to identify a sub-space in which a scrutinized sample is an outlier. Both are data-centered algorithms and consequently their computational complexity is prohibitive for processing data-streams.…”

Shapley Values of Reconstruction Errors of PCA for Explaining Anomaly Detection

“…These results do not completely preclude the use of reconstruction errors for explaining anomalies, but one should be aware that there are cases where the Shapley values do not completely agree with the reconstruction errors of each feature. Dang et al[22], Micenková et al[23], and Dang et al…”