FlyMC

Lukman, Jeffrey F.; Ke, Huan; Stuardo, Cesar A.; Suminto, Riza O.; Kurniawan, Daniar Heri; Simon, Dikaimin; Priambada, Satria; Tian, Chen; Ye, Feng; Leesatapornwongsa, Tanakorn; Gupta, Aarti; Lu, Shan; Gunawi, Haryadi S.

doi:10.1145/3302424.3303986

Cited by 17 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We use a simple static analysis from [Leesatapornwongsa et al 2014;Lukman et al 2019] to identify simple cases of commutative events. These include event handlers that count write acknowledgements or leader election votes, or that write a constant value to a variable, or that discard an event.…”

Section: Implementation Challengesmentioning

confidence: 99%

“…This is because Definition 2.1 labels a large number of events to be dependent, even though two events being processed on the same node may be commutative and hence be independent to each other. Therefore, in our implementation, we refine the dependency relation by employing the independence information extracted by the static analysis in [Leesatapornwongsa et al 2014;Lukman et al 2019]. Essentially, the static analysis provides the information of whether the execution of two events result in the same state.…”

Section: Computing the Set Of Racy Eventsmentioning

confidence: 99%

“…For example, if the update sets of the two events are the same (i.e., they write to the same variables) and the updates are increment operations. We refer the reader to [Lukman et al 2019] for more details of the static analysis.…”

Section: Computing the Set Of Racy Eventsmentioning

confidence: 99%

“…However, the number of events in an execution of Cassandra is large, resulting in a huge number of traces. For this benchmark, we strengthened notion of dependency using the independence of events identified by the static analysis in [Leesatapornwongsa et al 2014;Lukman et al 2019]. We used the refined dependency relation for all trace-aware algorithms.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…With a goal of detecting bugs due to the ordering of two events, Bita [Tasharofi et al 2013] covers only the pairwise orderings of the program events. Recent work [Leesatapornwongsa et al 2014;Liu et al 2017;Lukman et al 2019] reduces the state space by using some semantic information about the system under test. Different from the systematic testing approaches with a controlled scheduler, Falcon [Leners et al 2011] and Jepsen [Kingsbury 2018] modify the environment in which the system under test runs and stress test the system to expose faulty behaviors.…”

Section: Related Workmentioning

confidence: 99%

See 4 more Smart Citations

Trace aware random testing for distributed systems

Ozkan

Majumdar

Oraee

2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Distributed and concurrent applications often have subtle bugs that only get exposed under specific schedules. While these schedules may be found by systematic model checking techniques, in practice, model checkers do not scale to large systems. On the other hand, naive random exploration techniques often require a very large number of runs to find the specific interactions needed to expose a bug. In recent years, several random testing algorithms have been proposed that, on the one hand, exploit state-space reduction strategies from model checking and, on the other, provide guarantees on the probability of hitting bugs of certain kinds. These existing techniques exploit two orthogonal strategies to reduce the state space: partial-order reduction and bug depth. Testing algorithms based on partial order techniques, such as RAPOS or POS, ensure nonredundant exploration of independent interleavings among system events by imposing an equivalence relation on schedules and ideally exploring only one schedule from each equivalence class. Techniques based on bug depth, such as PCT, exploit the empirical observation that many bugs are exposed by the clever scheduling of a small number of key events. They bias the sample space of schedules to only cover all executions of small depth, rather than the much larger space of all schedules. At this point, there is no random testing algorithm that combines the power of both approaches. In this paper, we provide such an algorithm. Our algorithm, trace-aware PCT (taPCT), extends and unifies several different algorithms in the random testing literature. It samples the space of low-depth executions by constructing a schedule online, while taking dependencies among events into account. Moreover, the algorithm comes with a theoretical guarantee on the probability of sampling a trace of low depthÐthe probability grows exponentially with the depth but only polynomially with the number of racy events explored. We further show that the guarantee is optimal among a large class of techniques. We empirically compare our algorithm with several state-of-the-art random testing approaches for concurrent software on two large-scale distributed systems, Zookeeper and Cassandra, and show that our approach is effective in uncovering subtle bugs and usually outperforms related random testing algorithms. CCS Concepts: • Mathematics of computing → Combinatorics; • Software and its engineering → Software testing and debugging; • Theory of computation → Generating random combinatorial structures.

show abstract

Section: Implementation Challengesmentioning

confidence: 99%

Section: Computing the Set Of Racy Eventsmentioning

confidence: 99%

Section: Computing the Set Of Racy Eventsmentioning

confidence: 99%

Section: Experimental Evaluationmentioning

confidence: 99%