SimpleCAR: An Efficient Bug-Finding Tool Based on Approximate Reachability

Abstract

We present a new safety hardware model checker SimpleCAR that serves as a reference implementation for evaluating Complementary Approximate Reachability (CAR), a new SAT-based model checking framework inspired by classical reachability analysis. The tool gives a "bottom-line" performance measure for comparing future extensions to the framework. We demonstrate the performance of SimpleCAR on challenging benchmarks from the Hardware Model Checking Competition. Our experiments indicate that SimpleCAR is particularly suited for unsafety checking, or bug-finding; it is able to solve 7 unsafe instances within 1 h that are not solvable by any other state-of-the-art techniques, including BMC and IC3/PDR, within 8 h. We also identify a bug (reports safe instead of unsafe) and 48 counterexample generation errors in the tools compared in our analysis.

Intersection and Rotation of Assumption Literals Boosts Bug-Finding

“…A straightforward solution is to drop the literals in c one-by-one and check whether the UNSAT result is preserved. However, this solution is inefficient [23]; most attempts to find a smaller UC are unsuccessful and add to the overall runtime. Our heuristics: intersection and rotation can find smaller UC with negligible runtime overhead.…”

“…We incorporate the proposed heuristics in SimpleCAR [23]. Recall state enumeration in line 10 of Alg.…”

“…The common theme between these algorithms is that they are all SAT-based. BMC outperforms IMC on checking unsafe instances, i.e., bug-finding, while IC3/PDR and CAR can solve instances that BMC cannot [23]. It has been shown that better synergy between some of these algorithms and the SAT solver improves performance [14].…”

“…Complementary Approximate Reachability (CAR) [24,23] is a SAT-based model checking framework for reachability analysis. It can run in both forward and backward reachability modes; we focus on Backward-CAR as per previous work [23]. Contrary to reachability analysis via IC3/PDR, Backward-CAR maintains two sequences of over-and under-approximate reachable state-sets.…”

“…We argue that our heuristics are widely applicable and may improve the performance of other SAT-based model checking algorithms, like IC3/PDR. A thorough experimental evaluation on 748 single safety property benchmarks from HWMCC 2015 [2] and 2017 [3] reveals a 25% boost in the number of benchmarks that can be solved by Backward-CAR (155) compared to an earlier version in [23] (124). We also compare six implementations of Backward-CAR with varying heuristic combinations against reachability analysis algorithms (5×BMC, 9×IC3/PDR) in state-of-the-art model checking tools (ABC, nuXmv, IIMC, IC3Ref).…”