Combining Transformer with a Discriminator for Anomaly Detection in Multivariate Time Series

Maru, Chihiro; Brandherm, Boris; Kobayashi, Ichiro

doi:10.1109/scisisis55246.2022.10002131

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To our knowledge, ours is the first work to simplify and introduce this attention mechanism to multivariate temporal anomaly detection. Hence, we can obtain the values of all features at any moment t by a feedforward neural network (FNN) [11] based on historical serial time window data as…”

Section: Temporal Dependency Modelingmentioning

confidence: 99%

“…With its outstanding performance in sequence modeling and prediction tasks, numerous anomaly detection methods based on Transformer [9] have been proposed in recent years [10,11]. Wiederer incorporated an external attention mechanism on top of self-attention to model the correlation among multivariate time-series, and proposed a regularization-based method to constrain model parameters and prevent overfitting [12].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An enhanced abnormal information expression spatiotemporal model for anomaly detection in multivariate time-series

Ge,

Cheng,

Cao

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

The detection of anomalies in high-dimensional time-series has always played a crucial role in the domain of system security. Recently, with rapid advancements in transformer model and graph neural network (GNN) technologies, spatiotemporal modeling approaches for anomaly detection tasks have been greatly improved. However, most methods focus on optimizing upstream time-series prediction tasks by leveraging joint spatiotemporal features. Through experiments, we found that this modeling approach not only risks the loss of some original anomaly information during data preprocessing, but also focuses on optimizing the performance of the upstream prediction task and does not directly enhance the performance of the downstream detection task. We propose a spatiotemporal anomaly detection model that incorporates an improved attention mechanism in the process of temporal modeling. We adopt a heterogeneous graph contrastive learning approach in spatio modeling to compensate for the representation of anomalous behavioral information, thereby guiding the model through thorough training. Through validation on two widely used real-world datasets, we demonstrate that our model outperforms baseline methods. We also explore the impact of multivariate time-series prediction tasks on the detection task, and visualize the reasons behind the benefits gained by our model.

show abstract

Section: Temporal Dependency Modelingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An enhanced abnormal information expression spatiotemporal model for anomaly detection in multivariate time-series

Ge,

Cheng,

Cao

et al. 2024

Complex Intell. Syst.

View full text Add to dashboard Cite

show abstract

Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder

Liu,

Huang,

et al. 2024

Sensors

View full text Add to dashboard Cite

With the rapid development of industry, the risks factories face are increasing. Therefore, the anomaly detection algorithms deployed in factories need to have high accuracy, and they need to be able to promptly discover and locate the specific equipment causing the anomaly to restore the regular operation of the abnormal equipment. However, the neural network models currently deployed in factories cannot effectively capture both temporal features within dimensions and relationship features between dimensions; some algorithms that consider both types of features lack interpretability. Therefore, we propose a high-precision, interpretable anomaly detection algorithm based on variational autoencoder (VAE). We use a multi-scale local weight-sharing convolutional neural network structure to fully extract the temporal features within each dimension of the multi-dimensional time series. Then, we model the features from various aspects through multiple attention heads, extracting the relationship features between dimensions. We map the attention output results to the latent space distribution of the VAE and propose an optimization method to improve the reconstruction performance of the VAE, detecting anomalies through reconstruction errors. Regarding anomaly interpretability, we utilize the VAE probability distribution characteristics, decompose the obtained joint probability density into conditional probabilities on each dimension, and calculate the anomaly score, which provides helpful value for technicians. Experimental results show that our algorithm performed best in terms of F1 score and AUC value. The AUC value for anomaly detection is 0.982, and the F1 score is 0.905, which is 4% higher than the best-performing baseline algorithm, Transformer with a Discriminator for Anomaly Detection (TDAD). It also provides accurate anomaly interpretation capability.

show abstract

Combining Transformer with a Discriminator for Anomaly Detection in Multivariate Time Series

Cited by 2 publications

References 0 publications

An enhanced abnormal information expression spatiotemporal model for anomaly detection in multivariate time-series

An enhanced abnormal information expression spatiotemporal model for anomaly detection in multivariate time-series

Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder

Contact Info

Product

Resources

About