Coverage of Implementations by Simulating Specifications

Chockler, Hana; Kupferman, Orna

doi:10.1007/978-0-387-35608-2_34

Cited by 12 publications

(10 citation statements)

References 23 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The idea of this work was to evaluate the validity of formulas that are expressed on a State Machine. Chockler and Kupferman extended this work in [41] for coverage of Kripke structures.…”

Section: Related Workmentioning

confidence: 99%

Design by Contract to Improve Software Vigilance

Traon

Baudry

Jézéquel

2006

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Abstract-Design by Contract is a lightweight technique for embedding elements of formal specification (such as invariants, pre and postconditions) into an object-oriented design. When contracts are made executable, they can play the role of embedded, online oracles. Executable contracts allow components to be responsive to erroneous states and, thus, may help in detecting and locating faults. In this paper, we define Vigilance as the degree to which a program is able to detect an erroneous state at runtime. Diagnosability represents the effort needed to locate a fault once it has been detected. In order to estimate the benefit of using Design by Contract, we formalize both notions of Vigilance and Diagnosability as software quality measures. The main steps of measure elaboration are given, from informal definitions of the factors to be measured to the mathematical model of the measures. As is the standard in this domain, the parameters are then fixed through actual measures, based on a mutation analysis in our case. Several measures are presented that reveal and estimate the contribution of contracts to the overall quality of a system in terms of vigilance and diagnosability.

show abstract

“…The idea of this work was to evaluate the validity of formulas that are expressed on a State Machine. Chockler and Kupferman extended this work in [41] for coverage of Kripke structures.…”

Section: Related Workmentioning

confidence: 99%

Design by Contract to Improve Software Vigilance

Traon

Baudry

Jézéquel

2006

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…The authors of [10] improve the algorithm from [9] to be more efficient and general. The seminal work of Hoskote et al [9] was also extended to LTL model checking [5], to full CTL model checking [6], and to the simulation of specifications [4]. However, all these work are based on the state coverage metric.…”

Section: Previous Workmentioning

confidence: 99%

“…A well-studied metric is a purely state-based one [4,5,6,9], where an observed signal in a state is flipped (value toggled) to check if the satisfaction of any user-given property is affected (model checking result toggled). If the model checking result differs after toggling a signal value in a state, then the state is said to be covered.…”

Section: Mutation Coverage Estimationmentioning

confidence: 99%

Mutation Coverage Estimation for Model Checking

Lee

Hsiung

2004

Automated Technology for Verification and Analysis

View full text Add to dashboard Cite

Abstract. When engineers design a system, there is always a question about how exhaustive the system has been examined to be correct. Coverage estimation provides an answer to this question in testing. A model checker verifies a design exhaustively, and proves the satisfaction of property specifications. However, people have noticed that design errors exist even after model checking is done, which goes to show that the question "How complete is the model checking once done?" is still left relatively unaddressed by model checkers, except for some state-based coverage metrics and the coverage estimator for symbolic simulation in RED. As a more complete solution, we propose several structural mutation models and coverage metrics to cover different design aspects in a state graph and to estimate the completeness of model checking, respectively. Once a system state graph satisfies a given set of property specifications, we estimate the coverage of completeness for the set of properties by applying some mutations to the state graph and checking if the given set of properties is sensitive to the mutation. Our experiences on five application examples demonstrate how the proposed coverage estimation methodology helps verification engineers to find the uncovered hole.

show abstract

“…Both directions reason about a state-transition graph that models the system. The metric in [HKHZ99], later followed by [CKV01,CKKV01,CK02], is based on mutations applied to the graph. Essentially, a state s in the graph is covered by the specification if modifying the value of a variable in the state renders the specification untrue.…”

Section: Introductionmentioning

confidence: 99%

Sanity Checks in Formal Verification

Kupferman

2006

CONCUR 2006 – Concurrency Theory

Self Cite

View full text Add to dashboard Cite

Abstract.One of the advantages of temporal-logic model-checking tools is their ability to accompany a negative answer to the correctness query by a counterexample to the satisfaction of the specification in the system. On the other hand, when the answer to the correctness query is positive, most model-checking tools provide no additional information. In the last few years there has been growing awareness to the importance of suspecting the system or the specification of containing an error also in the case model checking succeeds. The main justification of such suspects are possible errors in the modeling of the system or of the specification. The goal of sanity checks is to detect such errors by further automatic reasoning. Two leading sanity checks are vacuity and coverage. In vacuity, the goal is to detect cases where the system satisfies the specification in some unintended trivial way. In coverage, the goal is to increase the exhaustiveness of the specification by detecting components of the system that do not play a role in verification process. For both checks, the challenge is to define vacuity and coverage formally, develop algorithms for detecting vacuous satisfaction and low coverage, and suggest methods for returning to the user helpful information. We survey existing work on vacuity and coverage and argue that, in many aspects, the two checks are essentially the same: both are based on repeating the verification process on some mutant input. In vacuity, mutations are in the specifications, whereas in coverage, mutations are in the system. This observation enables us to adopt work done in the context of vacuity to coverage, and vise versa.

show abstract

Coverage of Implementations by Simulating Specifications

Cited by 12 publications

References 23 publications

Design by Contract to Improve Software Vigilance

Design by Contract to Improve Software Vigilance

Mutation Coverage Estimation for Model Checking

Sanity Checks in Formal Verification

Contact Info

Product

Resources

About