An exploratory study of the evolution of communicated information about the execution of large software systems

Abstract

Abstract-A great deal of research in software engineering focuses on understanding the dynamic nature of software systems. Such research makes use of automated instrumentation and profiling techniques after fact, i.e., without considering domain knowledge. In this paper, we turn our attention to another source of dynamic information, i.e., the Communicated Information (CI) about the execution of a software system. Major examples of CI are execution logs and system events. They are generated from statements that are inserted intentionally by domain experts (e.g., developers or administrators) to convey crucial points of interest. The accessibility and domain-driven nature of the CI make it a valuable source for studying the evolution of a software system. In a case study on one large open source and one industrial software system, we explore the concept of CI and its evolution by mining the execution logs of these systems. Our study illustrates the need for better traceability techniques between CI and the Log Processing Apps that analyze the CI. In particular, we find that the CI changes at a rather high rate across versions, leading to fragile Log Processing Apps. 40% to 60% of these changes can be avoided and the impact of 15% to 50% of the changes can be controlled through the use of the robust analysis techniques by Log Processing Apps. We also find that Log Processing Apps that track implementation-level CI (e.g., performance analysis) are more fragile than Log Processing Apps that track domain-level CI (e.g., workload modeling), because the implementation-level CI is often short-lived.

Green mining: A methodology of relating software change to power consumption

“…Mining Software Repositories: Arguably any MSR work that is relevant to performance via traces is relevant to this work. For instance Shang et al [18] have investigated performance over versions of software, in particular Hadoop. They evaluated the multiple versions against the stability of the log messages, while not quite a performance measure it required running multiple versions of the system to collect this date.…”

Continuous validation of load test suites

“…However, we leave this to future work as automated instrumentation imposes a heavy overhead on the system [34]. Further, Shang et al report that execution logs are a rich source of information that are used by developers to convey important information about a system's behaviour [41]. Hence, automated instrumentation tools may not provide as deep an insight into the system's behaviour as execution logs.…”

GreenMiner: a hardware based mining software repositories software energy consumption framework

“…This is a dedicated hardware testbed meant to enable the task of green mining: the study of relationships between software change and software power use. While others such as Shang et al [19], Dyer et al [4], and Gousios et al [7] have leveraged cloud computing infrastructure such as Hadoop, mongodb, mysql and even bittorrent to store, distribute, collect, and execute large queries on MSR-relevant data, cloud computing is less than helpful to green mining. Virtualization and power estimation tend to rely on counter based models [9,1] to estimate supposed power use rather than measuring actual power use.…”