PADLA: A Dynamic Log Level Adapter Using Online Phase Detection

Abstract

Logging is an important feature for a software system to record its run-time information. Although detailed logs are helpful to identify the cause of a failure in a program execution, constantly recording detailed logs of a long-running system is challenging because of its performance overhead and storage cost. To solve the problem, we propose PADLA (Phase-Aware Dynamic Log Level Adapter) that dynamically adjusts the log level of a running system so that the system can record irregular events such as performance anomalies in detail while recording regular events concisely. PADLA is an extension of Apache Log4j, one of the most popular logging framework for Java. It employs an online phase detection algorithm to recognize irregular events. It monitors run-time performance of a system and learns regular execution phases of a program. If it recognizes a performance anomalies, it automatically changes the log level of a system to record the detailed behavior. In the case study, PADLA successfully recorded a detailed log for performance analysis of a server system under high load while suppressing the amount of log data and performance overhead.

A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis

“…Logging libraries, also called logging utilities (LU) [50] or logging libraries and utilities (LLUs), provide extra features, such as log level, which indicate the verbosity and the severity of the logging statement. Log levels help to better distinguish the importance of runtime events and control the number of logs collected on the storage device [51]. For example, less verbose levels, i.e., fatal, error, and warn, are leveraged to alarm the user when a potential problem happens in the system, and more verbose levels such as info, debug, and trace are utilized to record more general system events and information or detailed debugging.…”

“…As a result, maintaining and developing high-quality logging code as the feature code evolves remains challenging and crucial to the overall quality of the software. 4) The fourth challenge, and also the recent [51] and mostly overlooked in prior research [30], discusses the 1. J Empirical Software Engineering (12) EMSE [69], [75], [76], [77], [78], [79], [80], [81], [82], [83], [84], [85] 10.…”

“…One approach to tackle the challenge of the number of logs is to enable dynamic filtering of logs during runtime [51], [117]. Enablement of this paradigm would allow to take the pressure off the developers and enable more conservative addition of log statements in the source code, without being concerned about the overhead.…”

“…In addition to predicting log location, prior research has also investigated approaches for prediction of the appropriate logging verbosity level for newly composed logging statements [51], [80], [82], [88], [141]. The approaches either apply some type of learning to predict the log verbosity level [141], or perform dynamic adjustment of the log level during the runtime [51]. Other research also aims to predict the description [107], or variables included in the logging statements [67], [97].…”

“…An imperative trend that we foresee as future research is the need for adaptive logging [51]. On the one hand, continuously logging in details (e.g., in trace verbosity level) can infer performance overhead, and on the other hand, logging very little might degrade the effectiveness of the logs.…”