TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data

Abstract

Efficient anomaly detection and diagnosis in multivariate timeseries data is of great importance for modern industrial applications. However, building a system that is able to quickly and accurately pinpoint anomalous observations is a challenging problem. This is due to the lack of anomaly labels, high data volatility and the demands of ultra-low inference times in modern applications. Despite the recent developments of deep learning approaches for anomaly detection, only a few of them can address all of these challenges. In this paper, we propose TranAD, a deep transformer network based anomaly detection and diagnosis model which uses attentionbased sequence encoders to swiftly perform inference with the knowledge of the broader temporal trends in the data. TranAD uses focus score-based self-conditioning to enable robust multi-modal feature extraction and adversarial training to gain stability. Additionally, model-agnostic meta learning (MAML) allows us to train the model using limited data. Extensive empirical studies on six publicly available datasets demonstrate that TranAD can outperform state-of-the-art baseline methods in detection and diagnosis performance with data and time-efficient training. Specifically, TranAD increases F1 scores by up to 17%, reducing training times by up to 99% compared to the baselines.

LPC-AD: Fast and Accurate Multivariate Time Series Anomaly Detection via Latent Predictive Coding

“…To achieve this, many performance metrics of a cloud computing system such as CPU usage, disk I/O rate, network packet loss rate, etc., are monitored in real-time, which form a MTS [19,22,26]. From the MTS, system anomalies can be detected via machine learning [4,12,17,23,37,42]. which greatly improves troubleshooting of the system.…”

“…Real-world applications like cloud computing, micro-service systems, etc., generate large amount and high dimensional time series data and they needed to be processed by fast and accurate MTS anomaly detection methods. Although many machine learning algorithms were proposed to detect anomalies in MTS [4,12,17,23,29,37,42], how to achieve a fast training speed while retaining fairly high detection accuracy is underexplored. Classical methods like [6,14,20,29,39,41] have a fast training speed, but their detection accuracy is not high, due to low expressiveness capability of their model.…”

“…As a consequence, these sophisticated models becomes more and more time-consuming for training. Recently, the USAD [4] and TranAD [37] improve the training speed, while retaining a fairly high accuracy. These two algorithms follow the same architecture of adversarial training (refer to Section 2 for details) and their detection accuracy can be low in some datasets (refer to Section 6 for details).…”

“…It is challenging to design fast and accurate MTS anomaly detection algorithms. First, the MTS training data is usually not associated with anomalous labels and there is no specification on the pattern of an anomaly in general [4,37]. Second, the anomalous data points in MTS training data can mislead the trained model [18].…”

“…Second, the anomalous data points in MTS training data can mislead the trained model [18]. Third, the MTS data is usually of high dimension and it has complex spatial and temporal dependence [4,37], where the spatial dependence refers to the dependence among different time series. This makes it difficult to learn the normal and abnormal pattern from the MTS.…”