An Exploratory Study of the Evolution of Communicated Information about the Execution of Large Software Systems

Abstract

Abstract-A great deal of research in software engineering focuses on understanding the dynamic nature of software systems. Such research makes use of automated instrumentation and profiling techniques after fact, i.e., without considering domain knowledge. In this paper, we turn our attention to another source of dynamic information, i.e., the Communicated Information (CI) about the execution of a software system. Major examples of CI are execution logs and system events. They are generated from statements that are inserted intentionally by domain experts (e.g., developers or administrators) to convey crucial points of interest. The accessibility and domain-driven nature of the CI make it a valuable source for studying the evolution of a software system. In a case study on one large open source and one industrial software system, we explore the concept of CI and its evolution by mining the execution logs of these systems. Our study illustrates the need for better traceability techniques between CI and the Log Processing Apps that analyze the CI. In particular, we find that the CI changes at a rather high rate across versions, leading to fragile Log Processing Apps. 40% to 60% of these changes can be avoided and the impact of 15% to 50% of the changes can be controlled through the use of the robust analysis techniques by Log Processing Apps. We also find that Log Processing Apps that track implementation-level CI (e.g., performance analysis) are more fragile than Log Processing Apps that track domain-level CI (e.g., workload modeling), because the implementation-level CI is often short-lived.

Energy efficiency on the product roadmap: An empirical study across releases of a software product

“…Comparing aspects across releases is often discussed in terms of software evolution. 41 However, only few papers were found that investigate the EC of software and include a comparison between different releases. In 1 study 42 a comparison is made between 3 releases of rTorrent by "mining" EC and performance data.…”

Software energy profiling

“…EC comparison between releases: Comparing aspects across releases is often discussed in terms of software evolution [27]. However, only few papers were found that investigate the EC of software and include a comparison between different releases.…”

An exploratory study of the evolution of communicated information about the execution of large software systems

“…We use releases 0.14.0 to 0.21.0 for our study as shown in Table . We chose these releases because 0.14.0 is the earliest one that is able to run in our experimental environment and 0.21.0 is the most recent release at the time of our previous study . Among the studied releases, 0.20.1 and 0.20.2 are minor releases of the 0.20.0 series.…”

“…Studying CI only at execution level or code level is important because execution‐level CI contains the information that LPAs actually depend on and the code‐level CI has the potential to be communicated during execution. In our previous research , we only studied CI at execution level, but in this paper, we extend our study of CI by also performing source code analysis of logging statements.…”

“…In our previous research , we explored the evolution of CI by examining the logs of 10 releases of an open source software system ( Hadoop ) and 9 releases of a closed source large enterprise application, which we call EA . Because the CI during typical execution of a system may not cover all the CI that the system is capable of, this paper extends our research by performing a lower‐level study than the previous research.…”