Automatic Cause Detection of Performance Problems in Web Applications

Abstract

The execution of similar units can be compared by their internal behaviors to determine the causes of their potential performance issues. For instance, by examining the internal behaviors of different fast or slow web requests more closely, and by clustering and comparing their internal executions, one can determine what causes some requests to run slowly or behave in unexpected ways. In this paper, we propose a method of extracting the internal behavior of web requests as well as introduce a pipeline that detects performance issues in web requests and provides insights into their root causes. First, low-level and fine-grained information regarding each request is gathered by tracing both the user space and the kernel space. Second, further information is extracted and fed into an outlier detector. Finally, these outliers are then clustered by their behavior, and each group is analyzed separately. Experiments revealed that this pipeline is indeed able to detect slow web requests and provide additional insights into their true root causes. Notably, we were able to identify a real PHP cache contention issue using the proposed approach.

Combining Distributed and Kernel Tracing for Performance Analysis of Cloud Applications

“…Unlike SystemTap [29,30], ftrace [31] and eBPF [32], it does not suffer from a significant increase in latency when used on multicore systems [33]. LTTng saves trace data to disk-or sends it through the network-for offline analysis and relies on the Common Trace Format [34], a standard format built for high trace event throughput. It offers a flight recorder mode that reduces the overhead by flushing the buffered events to the disk only when the user requests a snapshot.…”

On Improving Deep Learning Trace Analysis with System Call Arguments

“…It is, however, a best practice to normalize the input vector to mitigate numerical instabilities, help training, and improve the model performance. Since the pid is not inherently meaningful in general 4 , any bijection from the argument space to a small interval such as [0, 1] or [−1, 1] works well. The simplest solution would be to map the pid uniformly to real values between [0, 1].…”

“…Most machine learning methods take a vector of numerical features as input. Hand-crafted features of traces have been proposed, but no representation seems to work universally well or to encapsulate the true underlying explanatory factors [2,3,4]. Instead of relying on hand-crafted features, neural networks learn how to extract meaningful features for the task.…”

Automated Cause Analysis of Latency Outliers Using System-Level Dependency Graphs

“…When used in combination with the bottleneck detector, it will pinpoint the lines of code responsible for the bottleneck. However, profilers are known to be ineffective because they operate by averaging the metrics, which may hide outliers [4].…”

“…In previous works, developers would use critical paths to observe how long each thread's state takes [4]. A critical path is a tool to display the execution states of a given thread at any particular time.…”