Analyzing multicore dumps to facilitate concurrency bug reproduction

Abstract

Debugging concurrent programs is difficult. This is primarily because the inherent non-determinism that arises because of scheduler interleavings makes it hard to easily reproduce bugs that may manifest only under certain interleavings. The problem is exacerbated in multi-core environments where there are multiple schedulers, one for each core. In this paper, we propose a reproduction technique for concurrent programs that execute on multi-core platforms. Our technique performs a lightweight analysis of a failing execution that occurs in a multi-core environment, and uses the result of the analysis to enable reproduction of the bug in a singlecore system, under the control of a deterministic scheduler.More specifically, our approach automatically identifies the execution point in the re-execution that corresponds to the failure point. It does so by analyzing the failure core dump and leveraging a technique called execution indexing that identifies a related point in the re-execution. By generating a core dump at this point, and comparing the differences betwen the two dumps, we are able to guide a search algorithm to efficiently generate a failure inducing schedule. Our experiments show that our technique is highly effective and has reasonable overhead.

Stride: Search-based deterministic replay in polynomial time via bounded linkage

“…Recognizing this drawback, the search-based replaying techniques( [7], [19]- [23]) do not record the exact RWlinkage and, instead, rely on the post-recording search to construct a feasible interleaving. The search-based replay techniques can incur a very low recording overhead 1 at the cost of losing the replay determinism.…”

Reconstructing Core Dumps

“…[31] analyzes multi-core dumps to reconstruct the thread schedule that caused a crash. Again, this assumes the program inputs are recorded, inducing overhead; however, RECORE does not handle concurrent programs at this point.…”

Isolating bugs in multithreaded programs using execution suppression

“…Much research work has been done to identify failure-triggering thread interleavings for inhouse testing [14,15] or for reproducing concurrency bugs that occur at the end-user side [16][17][18][19]. These approaches either provide a bug-triggering control to a runtime system, such as Valgrind or Pin, or they generate a whole program execution log, to ensure that a failure can be repeatedly reproduced for debugging.…”