”Have I Written Enough Properties?” - A Method of Comparison Between Specification and Implementation

Abstract

Abstract. This work presents a novel approach for evaluating the quality of the model checking process. Given a model of a design (or implementation) and a temporal logic formula that describes a specification, model checking determines whether the model satisfies the specification. Assume that all specification formulas were successfully checked for the implementation. Are we sure that the implementation is correct? If the specification is incomplete, we may fail to find an error in the implementation. On the other hand, if the specification is complete, then the model checking process can be stopped without adding more specification formulas. Thus, knowing whether the specification is complete may both avoid missed implementation errors and save precious verification time. The completeness of a specification with respect to a given implementation is determined as follows. The specification formula is first transformed into a tableau. The simulation preorder is then used to compare the implementation model and the tableau model. We suggest four comparison criteria, each revealing a certain dissimilarity between the implementation and the specification. If all comparison criteria are empty, we conclude that the tableau is bisimilar to the implementation model and that the specification fully describes the implementation. We also conclude that there are no redundant states in the implementation. The method is exemplified on a small hardware example. We implemented our method symbolically as an extension to SMV. The implementation involves efficient OBDD manipulations that reduce the number of OBDD variables from 4n to 2n.

Coverage of Implementations by Simulating Specifications

“…The algorithm we present in this section is symbolic, and it computes the relation C that is defined as follows. (w,v,w'): w,v E W,w' E W', and w' simulates win Iv,q}· Then, v is q-covered by S if there is w0 E W0 such that for all wb E W&, we have (wo, v, wb) ¢ C. Several symbolic simulation algorithms are described in the literature [McMillan, 1993;Henzinger et al, 1995;Katz et al, 1999]. We build our coverage algorithm on top of the straightforward symbolic implementation of Milner's fixed-point expression for simulation (see 8 in Section 2).…”

“…In order to do so, we use early quantification and variable interleaving in the OBDDs. Both techniques are used in [Katz et al, 1999] (see also [Katz, 2001]) in order to reduce the number of OBDD variables required for computing the simulation relation B from 4n to 2n.…”

“…In [Katz et al, 1999], early quantification is used for computing simulation as follows. Recall that B is the greatest fixed point of the expression 8 = { (w,w'): B0 (w,w') A Vu 3u': [R(w,u') A B(u,u') ,x) 1\g(z,x)).…”

“…There are two different approaches to coverage in model checking, where the specification is given as a temporal logic formula. One approach, introduced in Katz et al, 1999, states that a well-covered implementation should closely resemble the reduced tableau of its specification. Thus the coverage criteria of Katz et al are based on the analysis of the differences between the implementation and the tableau of its specification.…”

“…It has been recently shown in [Katz et al, 1999;Katz, 2001] how early quantification and variable interleaving in the OBDD can be used in order to reduce the number of required variables to 2n. Similarly, a naive implementation of the fixed-point expression with which the relation C is computed requires 4n + 2n' OBDD variables.…”