Supervised Autoencoder Variants for End to End Anomaly Detection

Lübbering, Max; Gebauer, Michael; Ramamurthy, Rajkumar; Sifa, Rafet; Bauckhage, Christian

doi:10.1007/978-3-030-68790-8_44

Cited by 4 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While ATA incorporates not only toxic but also non-toxic samples in the training procedure, OCA's training routine exposes the model only to toxic samples, leading to underperformance even compared to the random BASE baseline. This is a well-known problem which especially arises when the inliers correlate with outliers in feature space [3,4]. Interestingly, all methods perform superior on the insult test set, compared to the others.…”

Section: Resultsmentioning

confidence: 95%

“…In order to address this issue of diverse and previously unknown toxicity types, we present a comparative analysis of classification and outlier detection In this work, we consider three different types of methods for toxicity detection, namely a) representation learning based outlier detectors, b) ensemble methods and c) traditional deep neural networks. In the first case, a representation of the normal class (here, toxic class) is being learned and any sample that is very dissimilar from this representation is being rejected as an outlier [3,4,5,6]. In practice, this methodology has been successfully applied within a wide spectrum of domains, such as medicine [7], fraud detection [8] or intrusion detection [9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Toxicity Detection in Online Comments with Limited Data: A Comparative Analysis

Lübbering¹,

Pielka²,

Das³

et al. 2021

ESANN 2021 Proceedings

Self Cite

View full text Add to dashboard Cite

We present a comparative study on toxicity detection, focusing on the problem of identifying toxicity types of low prevalence and possibly even unobserved at training time. For this purpose, we train our models on a dataset that contains only a weak type of toxicity, and test whether they are able to generalize to more severe toxicity types. We find that representation learning and ensembling exceed the classification performance of simple classifiers on toxicity detection, while also providing significantly better generalization and robustness. All models benefit from a larger training set size, which even extends to the toxicity types unseen during training.

show abstract

Section: Resultsmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

Toxicity Detection in Online Comments with Limited Data: A Comparative Analysis

Lübbering¹,

Pielka²,

Das³

et al. 2021

ESANN 2021 Proceedings

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the related OCC setting with its focus on outlier detection, DNN-based approaches have been researched from three angles: (1) combining kernel methods [65] with DNN methods [14,60,71], (2) generative models (e.g., generative adversarial networks [22] or variational autoencoders [35])based outlier detectors [50,64,72], and (3) based on (semi-) supervised autoencoders [8,10,26,44,45,47]. Here, the key idea is to learn a representation of the inlier distribution and subsequently to estimate the outlierness of a sample via its reconstruction error.…”

Section: Related Workmentioning

confidence: 99%

“…Similar to existing autoencoder-based approaches for outlier detection [8,26,45], the decoupling autoencoder (DAE) method learns the outlierness of a sample via its reconstruction error. Existing approaches estimate the decision boundary via brute-force algorithms [1,45] or learn the decision boundary via a subsequent downstream layer [44]. In contrast, DAE learns the decision boundary end-to-end while optimizing for a pessimistic decision boundary that is most close to the inlier samples without compromising generalization performance.…”

Section: Decoupling Autoencodersmentioning

confidence: 99%

“…6 for ATA, each model has learned a representation of COI and rejects any major deviation from it as RC. While this indeed increases the robustness [44,45], it can also harm the classification performance since a representation for all input features needs to be learned [67]. Uninformative, non-causal features can therefore also have a diminishing effect on classification performance.…”

Section: Classification Concern Conflicting With Robustnessmentioning

confidence: 99%

See 1 more Smart Citation

Bounding open space risk with decoupling autoencoders in open set recognition

Lübbering

Gebauer

Ramamurthy

et al. 2022

Int J Data Sci Anal

Self Cite

View full text Add to dashboard Cite

One-vs-Rest (OVR) classification aims to distinguish a single class of interest (COI) from other classes. The concept of novelty detection and robustness to dataset shift becomes crucial in OVR when the scope of the rest class is extended from the classes observed during training to unseen and possibly unrelated classes, a setting referred to as open set recognition (OSR). In this work, we propose a novel architecture, namely decoupling autoencoder (DAE), which provides a proven upper bound on the open space risk and minimizes open space risk via a dedicated training routine. Our method is benchmarked within three different scenarios, each isolating different aspects of OSR, namely plain classification, outlier detection, and dataset shift. The results conclusively show that DAE achieves robust performance across all three tasks. This level of cross-task robustness is not observed for any of the seven potent baselines from the OSR, OVR, outlier detection, and ensembling domain which, apart from ATA (Lübbering et al., From imbalanced classification to supervised outlier detection problems: adversarially trained auto encoders. In: Artificial neural networks and machine learning—ICANN 2020, 2020), tend to fail on either one of the tasks. Similar to DAE, ATA is based on autoencoders and facilitates the reconstruction error to predict the inlierness of a sample. However unlike DAE, it does not provide any uncertainty scores and therefore lacks rudimentary means of interpretation. Our adversarial robustness and local stability results further support DAE’s superiority in the OSR setting, emphasizing its applicability in safety-critical systems.

show abstract