Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series

Soelch, Maximilian; Bayer, Justin; Ludersdorfer, Marvin; Smagt, Patrick van der

doi:10.48550/arxiv.1602.07109

Cited by 14 publications

(19 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively, researchers seek to use a deterministic Recurrent Neural Network (RNN) [29] to produce hidden states, then the latent variable is conditioned on the RNN hidden states and the previous latent variables, such that the temporal structure is captured. The resulting models include VRNN [16], STORN [30], which can be viewed as sequential versions of VAE. However, these models essentially hardly consider the presence of anomalies in their objective functions.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder

Yan

Wang

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

119

View full text Add to dashboard Cite

Deep generative models have demonstrated their effectiveness in learning latent representation and modeling complex dependencies of time series. In this paper, we present a Smoothness-Inducing Sequential Variational Auto-Encoder (SIS-VAE) model for robust estimation and anomaly detection of multi-dimensional time series. Our model is based on Variational Auto-Encoder (VAE), and its backbone is fulfilled by a Recurrent Neural Network to capture latent temporal structures of time series for both generative model and inference model. Specifically, our model parameterizes mean and variance for each timestamp with flexible neural networks, resulting in a non-stationary model that can work without the assumption of constant noise as commonly made by existing Markov models. However, such a flexibility may cause the model fragile to anomalies. To achieve robust density estimation which can also benefit detection tasks, we propose a smoothness-inducing prior over possible estimations. The proposed prior works as a regularizer that places penalty at non-smooth reconstructions. Our model is learned efficiently with a novel stochastic gradient variational Bayes estimator. In particular, we study two decision criteria for anomaly detection: reconstruction probability and reconstruction error. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.

show abstract

Section: Related Workmentioning

confidence: 99%

“…• STORN [30]. As a sequential version of VAE, the model is based on stochastic recurrent neural network (STORN).…”

Section: A Protocols and Settingsmentioning

confidence: 99%

Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder

Yan

Wang

et al. 2021

IEEE Trans. Neural Netw. Learning Syst.

119

View full text Add to dashboard Cite

show abstract

“…To detect outliers w.r.t. the target task, such approaches typically rely on some sort of confidence score to decide on the reliability of the prediction, which is either produced by modifying the model and/or training procedure, or computed/extracted post-hoc from the model and/or predictions (An & Cho, 2015;Sölch et al, 2016;Hendrycks & Gimpel, 2016;Liang et al, 2017;Hendrycks et al, 2018;Shafaei et al, 2018;DeVries & Taylor, 2018;Sricharan & Srivastava, 2018;Ahmed & Courville, 2019). Alternatively, some methods use predictive uncertainty estimates for OoD detection (Gal & Ghahramani, 2016;Lakshminarayanan et al, 2017;Malinin & Gales, 2018;Osawa et al, 2019;Ovadia et al, 2019) (cf.…”

Section: Related Workmentioning

confidence: 99%

Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection

Daxberger¹,

Hernández-Lobato²

2019

Preprint

View full text Add to dashboard Cite

Despite their successes, deep neural networks may make unreliable predictions when faced with test data drawn from a distribution different to that of the training data, constituting a major problem for AI safety. While this has recently motivated the development of methods to detect such outof-distribution (OoD) inputs, a robust solution is still lacking. We propose a new probabilistic, unsupervised approach to this problem based on a Bayesian variational autoencoder model, which estimates a full posterior distribution over the decoder parameters using stochastic gradient Markov chain Monte Carlo, instead of fitting a point estimate. We describe how informationtheoretic measures based on this posterior can then be used to detect OoD inputs both in input space and in the model's latent space. We empirically show the effectiveness of our approach.

show abstract

“…There are numerous papers describing the use of VAE for anomaly detection - [19], [22], [6], but none make a more complete comparison with other generative and classical methods and only some use non-image datasets. In [1], the authors describe the advantages of VAE over AE -it generalizes more easily since it is working on probabilities.…”

Section: Variational Autoencodermentioning

confidence: 99%

Are generative deep models for novelty detection truly better?

Škvára,

Pevný,

Šmídl

2018

Preprint

View full text Add to dashboard Cite

Many deep models have been recently proposed for anomaly detection. This paper presents comparison of selected generative deep models and classical anomaly detection methods on an extensive number of non-image benchmark datasets. We provide statistical comparison of the selected models, in many configurations, architectures and hyperparamaters. We arrive to conclusion that performance of the generative models is determined by the process of selection of their hyperparameters. Specifically, performance of the deep generative models deteriorates with decreasing amount of anomalous samples used in hyperparameter selection. In practical scenarios of anomaly detection, none of the deep generative models systematically outperforms the kNN.

show abstract

Variational Inference for On-line Anomaly Detection in High-Dimensional Time Series

Cited by 14 publications

References 0 publications

Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder

Anomaly Detection of Time Series With Smoothness-Inducing Sequential Variational Auto-Encoder

Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection

Are generative deep models for novelty detection truly better?

Contact Info

Product

Resources

About