Log Sequence Anomaly Detection Method Based on Contrastive Adversarial Training and Dual Feature Extraction

Abstract

The log messages generated in the system reflect the state of the system at all times. The realization of autonomous detection of abnormalities in log messages can help operators find abnormalities in time and provide a basis for analyzing the causes of abnormalities. First, this paper proposes a log sequence anomaly detection method based on contrastive adversarial training and dual feature extraction. This method uses BERT (Bidirectional Encoder Representations from Transformers) and VAE (Variational Auto-Encoder) to extract the semantic features and statistical features of the log sequence, respectively, and the dual features are combined to perform anomaly detection on the log sequence, with a novel contrastive adversarial training method also used to train the model. In addition, this paper introduces the method of obtaining statistical features of log sequence and the method of combining semantic features with statistical features. Furthermore, the specific process of contrastive adversarial training is described. Finally, an experimental comparison is carried out, and the experimental results show that the method in this paper is better than the contrasted log sequence anomaly detection method.

Deep Learning for Anomaly Detection in Log Data: A Survey

“…Due to their simplicity, their classification accuracies are usually outperformed by other deep learning models that are specifically designed to capture common characteristics present in sequential data. Accordingly, they are rarely considered in the reviewed literature and only occur in combination with other deep learning models or as supplementary attention mechanisms [1], [45].…”

“…Any input data that is fed into an already trained network and yields a high reconstruction error is then considered as anomalous. Besides the standard model for Autoencoders, there are also several related types, such as Variational Autoencoders (VAE) that operate on statistical distributions [1], [52], [58], Conditional Variational Autoencoders (CVAE) that add conditional information such as event types to the training [49], and Convolutional Autoencoder (CAE) that leverage the advantages of CNNs regarding learning of location-independent features [58].…”

“…They actually consist of two separate components that compete with each other: a generator that produces new data that resembles the input data, and a discriminator that estimates the probability that some data stems from the input data, which is used to improve the generator. Existing approaches use different models to construct GANs, including LSTM RNNs [41], [66], CNNs in combination with GRUs [35], and Transformers [1], [76].…”

“…However, some authors also use combinations of different models. For example, Wang et al [1] propose to use a MLP to combine the output of a VAE with that of a Transformer trained with adversarial learning. 2) Training loss function: Loss functions (DL-2) are essential for training of deep neural networks as they quantify the difference between the output of the neural network and the expected result [80].…”

“…Some authors also use custom embedding models based on deep learning; we refer to their output as Deep Encoded Embeddings (DE). This includes a combination of character-, event-and sequence-based embeddings [42], attention mechanisms using MLPs and CNNs [45], and token counts with label information fed into VAEs [1].…”