DepGraph: Localizing Performance Bottlenecks in Multi-Core Applications Using Waiting Dependency Graphs and Software Tracing

Abstract

This paper addresses the challenge of understanding the waiting dependencies between the threads and hardware resources required to complete a task. The objective is to improve software performance by detecting the underlying bottlenecks caused by system-level blocking dependencies. In this paper, we use a system level tracing approach to extract a Waiting Dependency Graph that shows the breakdown of a task execution among all the interleaving threads and resources. The method allows developers and system administrators to quickly discover how the total execution time is divided among its interacting threads and resources. Ultimately, the method helps detecting bottlenecks and highlighting their possible causes. Our experiments show the effectiveness of the proposed approach in several industry-level use cases. Three performance anomalies are analysed and explained using the proposed approach. Evaluating the method efficiency reveals that the imposed overhead never exceeds 10.1%, therefore making it suitable for in-production environments.

Automated Cause Analysis of Latency Outliers Using System-Level Dependency Graphs

“…Finally, we use our proposed merging and comparison algorithms to find the areas contributing to the latency issues. Our work in this paper expands upon what was presented by Ezzati-Jivan et al [1]. The authors discuss converting system-level traces into DepGraphs to reduce the effort required to analyze raw trace data.…”

“…For example, the critical path of a thread that is waiting for another thread to finish would not display the details of that other thread. Dependency graphs, proposed by Ezzati-Jivan et al [1] and adapted in our work, can show all thread interactions and help identify bottlenecks.…”

“…Our proposed method uses Waiting-Dependency Graphs (Dep-Graphs) to illustrate the interactions between different threads and system resources (i.e., disk, CPU, network, etc.). Unlike Ezzati-Jivan et al [1], however, we aim to avoid manual analysis of these DepGraphs. Instead, we want to automate the process of detecting outliers within a dataset of DepGraphs.…”

“…Analyses are conducted either on-CPU or off-CPU depending on the type of issue [1,2]. When the analysis is conducted on-CPU, any threads running on the CPU can be analyzed.…”

“…To automate the task, we first filter out the parts of the trace that do not correspond to the program. Using the filtered trace data, we then construct Waiting-Dependency Graphs (DepGraphs) [1], displaying the internal interactions between threads and system resources. Next, we use graph embedding techniques to convert the DepGraphs into fixed-size vectors compatible with most offthe-shelf machine learning algorithms.…”