Log File Analysis Based on Machine Learning: A Survey

Abstract

In the past few years, software monitoring and log analysis become very interesting topics because it supports developers during software developing, identify problems with software systems and solving some of security issues. A log file is a computer-generated data file which provides information on use patterns, activities, and processes occurring within an operating system, application, server, or other devices. The traditional manual log inspection and analysis became impractical and almost impossible due logs’ nature as unstructured, to address this challenge, Machine Learning (ML) is regarded as a reliable solution to analyze log files automatically. This survey tries to explore the existing ML approaches and techniques which are utilized in analyzing log file types. It retrieves and presents the existing relevant studies from different scholar databases, then delivers a detailed comparison among them. It also thoroughly reviews utilized ML techniques in inspecting log files and defines the existing challenges and obstacles for this domain that requires further improvements.

Analysis of log files to enable smart-troubleshooting in Industry 4.0: a systematic mapping study

“…This survey covers various aspects of log analysis, including prediction, reporting, profiling, anomaly detection, and user behavior. The authors also discuss the challenges and obstacles in this domain that require further improvements [42].…”