OneLog: Towards End-to-End Training in Software Log Anomaly Detection

Abstract

In recent years, with the growth of online services and IoT devices, software log anomaly detection has become a significant concern for both academia and industry. However, at the time of writing this paper, almost all contributions to the log anomaly detection task, follow the same traditional architecture based on parsing, vectorizing, and classifying.This paper proposes OneLog, a new approach that uses a large deep model based on instead of multiple small components. OneLog utilizes a character-based convolutional neural network (CNN) originating from traditional NLP tasks. This allows the model to take advantage of multiple datasets at once and take advantage of numbers and punctuations, which were removed in previous architectures.We evaluate OneLog using four open data sets Hadoop Distributed File System (HDFS), BlueGene/L (BGL), Hadoop, and Open-Stack. We evaluate our model with single and multi-project datasets. Additionally, we evaluate robustness with synthetically evolved datasets and ahead-of-time anomaly detection test that indicates capabilities to predict anomalies before occurring.To the best of our knowledge, our multi-project model outperforms state-of-the-art methods in HDFS, Hadoop, and BGL datasets, respectively setting getting 𝐹 1 scores of 99.99, 99.99, and 99.98. However, OneLog's performance on the Openstack is unsatisfying with 𝐹 1 score of only 21.18. Furthermore, Onelogs performance suffers very little from noise showing 𝐹 1 scores of 99.95, 99.92, and 99.98 in HDFS, Hadoop, and BGL. Our work demonstrates that character-level CNNs can successfully utilize multiple datasets that boost learning in log anomaly detection. Our 𝐹 1 scores for HDFS, Hadoop, and BGL are almost too good to be true, thus upon acceptance of this paper, we offer our code for further replications. Finally, future work is needed in investigating the poor performance of the Openstack dataset.

Deep Learning for Anomaly Detection in Log Data: A Survey

“…Guo et al [38] are the only authors to consider federated learning, where learning takes place in a distributed manner across multiple systems. Hashemi et al [42] also go into this direction as they combine multiple data sets to evaluate whether this affects the performance of their model. We believe that federated learning could be an interesting topic for future publications as there exist many real-world scenarios where log data is monitored in distributed machines but orchestration of deployed detectors takes place centrally [106].…”

“…Some authors also use custom embedding models based on deep learning; we refer to their output as Deep Encoded Embeddings (DE). This includes a combination of character-, event-and sequence-based embeddings [42], attention mechanisms using MLPs and CNNs [45], and token counts with label information fed into VAEs [1].…”

“…comp. Failures [1], [17], [20], [22], [23], [25], [27], [29]- [31], [33], [37]- [40], [42], [44], [46]- [48], [51], [52], [55], [56], [58], [59], [61], [62], [65]- [74], [76]- [78] BlueGene/L (BGL) [89] 2007 High-perf. comp.…”

“…-Failures [24], [34]- [36], [38]- [41], [55], [60], [64], [71], [73] OpenStack [20] 2017 Virtual machines Failures [20], [26], [34]- [36], [42], [44], [50], [52], [63], [76]- [78] Hadoop [90] 2016 High-perf. comp.…”

“…Other metrics that are more specific to deep learning applications are the number of model parameters [38], [61] and time to train models or run the detection (ER-3) [29], [32], [37], [47], [52], [68]. Some authors also assess characteristics of their approaches that go beyond standard anomaly detection evaluations, for example, whether training on combinations of multiple data sets improves the overall performance of classification [42] or whether their approaches are robust against changes of log patterns over time [17], [42], [44].…”