An overview and a benchmark of active learning for outlier detection with one-class classifiers

Trittenbach, Holger; Englhardt, Adrian; Böhm, Klemens

doi:10.1016/j.eswa.2020.114372

Cited by 29 publications

(6 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They are subsequently updated by incorporating labeled points from feedback into the learning problem. In [17], the authors used variants of SSAD model for benchmarking and found that there is no one-fit-all strategy for one-class active learning. Recently, Amazon developed NCAD based on deep semi-supervised learning [14] for time series anomaly detection [3].…”

Section: Related Workmentioning

confidence: 99%

Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

Bodor¹,

Hoang²,

Jiang³

2022

Preprint

View full text Add to dashboard Cite

Key Performance Indicators (KPI), which are essentially time series data, have been widely used to indicate the performance of telecom networks. Based on the given KPIs, a large set of anomaly detection algorithms have been deployed for detecting the unexpected network incidents. Generally, unsupervised anomaly detection algorithms gain more popularity than the supervised ones, due to the fact that labeling KPIs is extremely time-and resource-consuming, and error-prone. However, those unsupervised anomaly detection algorithms often suffer from excessive false alarms, especially in the presence of concept drifts resulting from network re-configurations or maintenance. To tackle this challenge and improve the overall performance of unsupervised anomaly detection algorithms, we propose to use active learning to introduce and benefit from the feedback of operators, who can verify the alarms (both false and true ones) and label the corresponding KPIs with reasonable effort. Specifically, we develop three query strategies to select the most informative and representative samples to label. We also develop an efficient method to update the weights of Isolation Forest and optimally adjust the decision threshold, so as to eventually improve the performance of detection model. The experiments with one public dataset and one proprietary dataset demonstrate that our active learning empowered anomaly detection pipeline could achieve performance gain, in terms of F1-score, more than 50% over the baseline algorithm. It also outperforms the existing active learning based methods by approximately 6% − 10%, with significantly reduced budget (the ratio of samples to be labeled).

show abstract

Section: Related Workmentioning

confidence: 99%

Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

Bodor¹,

Hoang²,

Jiang³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Model-based techniques can be divided into (i) models that learn and predict whether the value is anomalous and (ii) models that compare the potential outlier with expected values drawn from a generative model or data distribution. Since model-based techniques require labeled data, active learning can be utilized to minimize the labeling effort [57]. Among the models of the first group, we find the SVM-based models, such as the one-class support vector machine (OC-SVM), which was introduced by [58], and later enhanced by many authors [59,60].…”

Section: Time Series Anomaly Detectionmentioning

confidence: 99%

Explaining Bad Forecasts in Global Time Series Models

et al. 2021

View full text Add to dashboard Cite

While increasing empirical evidence suggests that global time series forecasting models can achieve better forecasting performance than local ones, there is a research void regarding when and why the global models fail to provide a good forecast. This paper uses anomaly detection algorithms and explainable artificial intelligence (XAI) to answer when and why a forecast should not be trusted. To address this issue, a dashboard was built to inform the user regarding (i) the relevance of the features for that particular forecast, (ii) which training samples most likely influenced the forecast outcome, (iii) why the forecast is considered an outlier, and (iv) provide a range of counterfactual examples to understand value changes, in the feature vector or the predicted value, can lead to a different outcome. Moreover, a modular architecture and a methodology were developed to iteratively remove noisy data instances from the train set, to enhance the overall global time series forecasting model performance. Finally, to test the effectiveness of the proposed approach, it was validated on two publicly available real-world datasets.

show abstract

“…We roughly group all the methods into: traditional methods and deep methods. More comprehensive review can be found in [1], [27], [28].…”

Section: Anomaly Detectionmentioning

confidence: 99%

Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning

Chen¹,

Duan

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Anomaly detection has wide applications in machine intelligence but is still a difficult unsolved problem. Major challenges include the rarity of labeled anomalies and it is a class highly imbalanced problem. Traditional unsupervised anomaly detectors are suboptimal while supervised models can easily make biased predictions towards normal data. In this paper, we present a new supervised anomaly detector through introducing the novel Ensemble Active Learning Generative Adversarial Network (EAL-GAN). EAL-GAN is a conditional GAN having a unique one generator vs. multiple discriminators architecture where anomaly detection is implemented by an auxiliary classifier of the discriminator. In addition to using the conditional GAN to generate class balanced supplementary training data, an innovative ensemble learning loss function ensuring each discriminator makes up for the deficiencies of the others is designed to overcome the class imbalanced problem, and an active learning algorithm is introduced to significantly reduce the cost of labeling real-world data. We present extensive experimental results to demonstrate that the new anomaly detector consistently outperforms a variety of SOTA methods by significant margins. The codes are available on Github.

show abstract

An overview and a benchmark of active learning for outlier detection with one-class classifiers

Cited by 29 publications

References 18 publications

Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

Little Help Makes a Big Difference: Leveraging Active Learning to Improve Unsupervised Time Series Anomaly Detection

Explaining Bad Forecasts in Global Time Series Models

Supervised Anomaly Detection via Conditional Generative Adversarial Network and Ensemble Active Learning

Contact Info

Product

Resources

About