History-Aware Data Structure Repair Using SAT

Abstract

Abstract. Data structure repair corrects erroneous executions in deployed programs while they execute, eliminating costly downtime. Recent techniques show how to leverage specifications and a SAT solver to enforce specification conformance at runtime. While this powerful methodology increases the reliability of deployed programs, scalability remains a key technical challenge-satisfying a specification often results in the exploration of a huge state space. We present a novel technique, called history-aware contract-based repair for more efficient data structure repair using SAT. Our insight is two-fold: (1) the dynamic program trace of field writes and reads provides useful guidance to repair incorrect state mutations by a faulty program; and (2) we show how to execute SAT using unsatisfiable cores it generates, in an efficient iterative approach on successive problems with increasing state spaces, in order to utilize the history of previous runs as captured in the unsatisfiable core. We implement this approach in a new tool, called Cobbler, that repairs Java programs. Experimental results on two large applications and a library implementation of a linked list show that Cobbler significantly outperforms previous techniques for specification-based repair using SAT, and finds and repairs a previously undetected bug.

Training Binary Classifiers as Data Structure Invariants

“…The results are summarized in Table III. RQ3 is the only research question that does not demand a provided class invariant for assessment. To evaluate it, we took buggy implementations of data structures from the literature: the scheduler implementation from the SIR repository [12], an implementation of n-ary trees that is part of the ANTLR parser generator, implementations of routines of a set of integers, over red black trees, with seeded bugs, presented in [42], binary search trees and binomial heaps used in the empirical evaluation in [14] containing one real bug each, and a fibonacci heap implementation taken from [1], containing a real bug. For each case study, we took a set of builders, and generated tests with Randoop from which we learned an object classifier with our technique, with a relatively small scope (5 for all cases), and produced likely invariants with Daikon, processed as for RQ2.…”

“…We did not include this case study in our evaluation because all seeded bugs correspond to routines that do not change object states, and thus cannot be caught by just checking invariant preservation. The bugs seeded in the red-black tree implementation from [42] all correspond to the insert method. We trained the neural network using a correct version of this method, and then used it to attempt to catch the seeded bugs.…”

“…In this context, we compare our technique with Daikon [13], a tool for dynamic invariant discovery. Finally, we also compare our object classifiers with invariants produced by Daikon, in bug finding through test generation, for a number of case studies involving data structures, taken from the literature: schedule from the SIR repository [12], an implementation of n-ary trees in the ANTLR parser generator, a redblack tree implementation of integer sets introduced in [42], binary search trees and binomial heaps from the evaluation performed in [14], and fibonacci heaps from the graphmaker library [1]. Our experiments show that our mechanism for producing invalid inputs is effective, and that the learned classifiers achieve significantly better accuracy in identifying valid/invalid instances, compared to Daikon (high precision and recall both in negative and positive cases).…”

Augmenting Field Data for Testing Systems Subject to Incremental Requirements Changes

“…Note that maximising test suite code coverage is a common requirement in industry and for our partner, and is therefore an important aspect to consider. We used a code coverage tool, EclEmma [Mountainminds 2006], to measure and evaluate code coverage in terms of bytecode instructions and branches. An analysis by Li et al [2013] has determined that the implementation of branch coverage in EclEmma, which measures the branches covered at the bytecode level, provides the equivalent of clause coverage (i.e.…”