Efficient and deterministic record & replay for actor languages

Abstract

With the ubiquity of parallel commodity hardware, developers turn to high-level concurrency models such as the actor model to lower the complexity of concurrent software. However, debugging concurrent software is hard, especially for concurrency models with a limited set of supporting tools. Such tools often deal only with the underlying threads and locks, which obscures the view on e.g. actors and messages and thereby introduces additional complexity.To improve on this situation, we present a low-overhead record & replay approach for actor languages. It allows one to debug concurrency issues deterministically based on a previously recorded trace. Our evaluation shows that the average run-time overhead for tracing on benchmarks from the Savina suite is 10% (min. 0%, max. 20%). For Acme-Air, a modern web application, we see a maximum increase of 1% in latency for HTTP requests and about 1.4 MB/s of trace data. These results are a first step towards deterministic replay debugging of actor systems in production.

Asynchronous snapshots of actor systems for latency-sensitive applications

“…Integration with Record & Replay Our snapshotting approach is integrated with the SOMns record & replay implementation [3]. Both components can be used individually or together.…”

Global Reproducibility Through Local Control for Distributed Active Objects

“…This technique is typically used for rich debuggers like Actoverse [24] for Scala's Akka library, which provides visualization support similar to ours. However, content-based approaches do not scale well [27], because the traces can become very large for message-intensive applications.…”

“…MPI assumes that messages from the same source are received in order, this does not generally hold for actor systems. Aumayr et al in [27] study ordering-based replay for actor systems with a memory-efficient representation of the generated traces. Netzer et al [29] propose an interesting method to only trace events directly related to races, rather than all events (removing up to 99% of the events).…”

MReplayer

“…The ARB takes the initial transition 1 and waits in the state 1 for a message request from PH_1 or PH_2 or PH_3. Upon reception of the request messages, the transition 2 is taken and the ARB examines whether forks in both sides of the philosopher are free (e.g., for PH_1 forks[0] and forks [2] are checked). If so, the ARB sends reply(true) and updates variable forks accordingly (e.g., for PH_1 forks[0]=1 and forks[2]=1).…”

“…Tveito et al [20] propose an approach for record and replay for actor languages that reproduces a specific run for non-deterministic actor systems. Similarly, Aumayer et al [2] introduce deterministic record and replay for actor languages that reduce the overhead on nodes. However, neither approach directly supports the implementation of a centralized debugger.…”