A New Distributed Log Anomaly Detection Method based on Message Middleware and ATT-GRU

Abstract

Logs play an important role in mastering the health of the system, experienced operation and maintenance engineer can judge which part of the system has a problem by checking the logs. In recent years, many system architectures have changed from single application to distributed application, which leads to a very huge number of logs in the system and manually check the logs to find system errors impractically. To solve the above problems, we propose a method based on Message Middleware and ATT-GRU (Attention Gate Recurrent Unit) to detect the logs anomaly of distributed systems. The works of this paper mainly include two aspects: (1) We design a high-performance distributed logs collection architecture to complete the logs collection of the distributed system. (2)We improve the existing GRU by introducing the attention mechanism to weight the key parts of the logs sequence, which can improve the training efficiency and recognition accuracy of the model to a certain extent. The results of experiments show that our method has better superiority and reliability.