Incorporating Expert Feedback into Active Anomaly Discovery

Abstract

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomalies are false positives and not interesting from the application perspective. In this paper, we aim to make the analyst's job easier by allowing for analyst feedback during the investigation process. Ideally, the feedback influences the ranking of the anomaly detector in a way that reduces the number of false positives that must be examined before discovering the anomalies of interest. In particular, we introduce a novel technique for incorporating simple binary feedback into tree-based anomaly detectors. We focus on the Isolation Forest algorithm as a representative tree-based anomaly detector, and show that we can significantly improve its performance by incorporating feedback, when compared with the baseline algorithm that does not incorporate feedback. Our technique is simple and scales well as the size of the data increases, which makes it suitable for interactive discovery of anomalies in large datasets.

LSCP: Locally Selective Combination in Parallel Outlier Ensembles

“…To induce diversity among base detectors, distinct initialization hyperparameters-specifically the number of neighbors (M inP ts) used in each LOF detector-are randomly selected in the range of [5,200]. It is noted that a narrower range (i.e., [10,150]) is taken for certain datasets due to limitations associated with size or computational cost. For GG AOM and GG MOA, the Tables 2 and 3 summarize the ROC-AUC and mAP scores on the 20 datasets.…”

“…Since the ground truth is often absent in outlier mining [1], unsupervised detection methods are commonly used for this task [5,8,17]. However, unsupervised approaches are susceptible to generating high false positive and false negative rates [10]. To improve model accuracy and stability in these scenarios, recent research explores ensemble approaches to outlier detection [1,3,24,32].…”

Crowdsourced Wireless Spectrum Anomaly Detection

“…A few semi-supervised anomaly detection algorithms exist in literature such as Lightweight on-line detector of anomalies (LODA) [18,19]. In [18], the authors make use of an unsupervised ensemble-based algorithm which is further enhanced with an active learning phase in [19]. A semi-supervised accuracy-at-the-top loss function is used to update weights of LODA based on user feedback.…”

“…A semi-supervised accuracy-at-the-top loss function is used to update weights of LODA based on user feedback. The base anomaly detection algorithm is not limited to LODA and the authors also show that the active anomaly detection phase can be incorporated with tree based algorithms too [20]. On the other hand in [21], the authors formulate anomaly detection as an optimization problem and propose a generalization using both labeled and unlabeled instances.…”

Extended Isolation Forest

“…Rather, it utilizes the fact that anomalous data are "few and different". Most anomaly detection algorithms find anomalies by understanding the distribution of their properties and isolating them from the rest of normal data samples [4], [13], [5]. In an Isolation Forest, data is sub-sampled, and processed in a tree structure based on random cuts in the values of randomly selected features in the dataset.…”