Runtime latency detection and analysis

Abstract

Summary Detecting latency‐related problems in production environments is usually carried out at the application level with custom instrumentation. This is enough to detect high latencies in instrumented applications but does not provide all the information required to understand the source of the latency and is dependent on manually deployed instrumentation. The abnormal latencies usually start in the operating system kernel because of contention on physical resources or locks. Hence, finding the root cause of a latency may require a kernel trace. This trace can easily represent hundreds of thousands of events per second. In this paper, we propose and evaluate a methodology, efficient algorithms, and concurrent data structures to detect and analyze latency problems that occur at the kernel level. We introduce a new kernel‐based approach that enables developers and administrators to efficiently track latency problems in production and trigger actions when abnormal conditions are detected. The result of this study is a working scalable latency tracker and an efficient approach to perform stateful tracing in production. Copyright © 2016 John Wiley & Sons, Ltd.

Combining Distributed and Kernel Tracing for Performance Analysis of Cloud Applications

“…It offers a flight recorder mode that reduces the overhead by flushing the buffered events to the disk only when the user requests a snapshot. Desfossez et al [35] uses this feature for efficient latency analysis of applications.…”

DepGraph: Localizing Performance Bottlenecks in Multi-Core Applications Using Waiting Dependency Graphs and Software Tracing

“…Software performance analysis has been widely studied in the literature. One approach to evaluating software performance is to focus on metrics such as the number of external interrupts [15], the number of scheduling per second [16], the number of processes waiting for disk blocks [12], and the critical section pressure [17] to identify the factors that affect performance. Giraldeau et al [16] describe how runtime metrics about the processor, memory, file systems, and network usage, collected from Linux kernel traces allow diagnosing latency.…”

Predicting temporal violations for parallel business cloud workflows

“…In addition, as a way of detecting and handling runtime latency in application, Desfossez et al proposed a novel methodology and concurrent data structures to detect and analyze latency problems that occur at the kernel level. This kernel‐based approach enables developers and administrators to efficiently track latency problems in production and trigger actions when abnormal conditions are detected . In the work of Cheikhrouhou et al, the authors presented a novel checking–based verification approach to handle the verification problem of timed processes.…”