Proof-based coverage metrics for formal verification

Ghassabani, Elaheh; Gacek, Andrew; Whalen, Michael W.; Heimdahl, Mats P. E.; Wagner, Lucas

doi:10.1109/ase.2017.8115632

Cited by 15 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This algorithm shares some similarities with one by Chockler et al [8] for determining non-deterministic mutation coverage (Nondet-Cov). This is expected since Nondet-Cov is equivalent to coverage based on MUST elements, as proved by Ghassabani et al [11].…”

Section: Computing the Must Setmentioning

confidence: 53%

“…This categorization provides complete traceability between specification and design elements, and can be used for coverage analysis [11] and tracking the safety impact of model changes. For instance, a change to one of the elements in MAY (S, P ) will not affect the satisfaction of P but will definitely impact some other property Q if it occurs in MUST (S, Q).…”

Section: Ivcs For Coverage and Change Impact Analysismentioning

confidence: 99%

“…In general, proof-based traceability information can be used to perform a variety of engineering analyses, including vacuity detection [14]; coverage analysis [8,11]; impact analysis [16], design optimization; and robustness analysis [18,20]. Identifying which model elements are required for a proof, and assessing the relative importance of different model elements is critical to determine the quality of the overall model (including its assume-guarantee specification), determining when and where to implement changes, identifying components that need to be reverified, and measure the tolerance and resilience of the system against faults and attacks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Merit and Blame Assignment with Kind 2

Larraz¹,

Laurent²,

Tinelli³

2021

Preprint

View full text Add to dashboard Cite

We introduce two new major features of the open-source model checker Kind 2 which provide traceability information between specification and design elements such as assumptions, guarantees, or other behavioral constraints in synchronous reactive system models. This new version of Kind 2 can identify minimal sets of design elements, known as Minimal Inductive Validity Cores, which are sufficient to prove a given set of safety properties, and also determine the set of MUST elements, design elements that are necessary to prove the given properties. In addition, Kind 2 is able to find minimal sets of design constraints, known as Minimal Cut Sets, whose violation leads the system to an unsafe state. The computed information can be used for several purposes, including assessing the quality of a system specification, tracking the safety impact of model changes, and analyzing the tolerance and resilience of a system against faults or cyber-attacks. We describe these new capabilities in some detail and report on an initial experimental evaluation of some of them.

show abstract

Section: Computing the Must Setmentioning

confidence: 53%

Section: Ivcs For Coverage and Change Impact Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Merit and Blame Assignment with Kind 2

Larraz¹,

Laurent²,

Tinelli³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In this section we re-use and adapt the formalization of IVC given by Ghassabani et al in [14] to compare IVC to our mutation proof using similar definitions of coverage.…”

Section: Ivc Formalizationsmentioning

confidence: 99%

“…Property and Model coverage. The article by Ghassabani et al [14] defines the two following metrics of coverage.…”

Section: Ivc Formalizationsmentioning

confidence: 99%

Specification Quality Metrics Based on Mutation and Inductive Incremental Model Checking

Todorov

Taha

Boulanger

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

When using formal verification on Simulink or SCADE models, an important question about their certification is how well the specified properties cover the entire model. A method using unsatisfiable cores and inductive model checking called IVC (Inductive Validity Cores) has been recently proposed within modern SMT-based model checkers such as JKind. The IVC algorithm determines a minimal set of model elements necessary to establish a proof and gives back the traceability to the design elements (lines of code) necessary for the proof. These metrics are interesting but are rather coarse grain for certification purposes. In this paper, we propose to use mutation combined with incremental inductive model checking to give more precision and quality to the traceability process and look inside the lines of code. Our algorithm, based on the result of IVC, mutates the source code to determine which parts inside a line of code have an impact on the properties (killed mutants) and which parts have no impact on the properties (survived mutants). Furthermore, using the incremental feature present in modern SMT-solvers, we observe that mutation can scale up to industrial models. We demonstrate the metrics first on a simple example, then on a complex industrial program and on the JKind benchmark.

show abstract

Approximate Counting of Minimal Unsatisfiable Subsets

Bendík

Meel

2020

Computer Aided Verification

View full text Add to dashboard Cite

Given an unsatisfiable formula F in CNF, i.e. a set of clauses, the problem of Minimal Unsatisfiable Subset (MUS) seeks to identify a minimal subset of clauses such that N is unsatisfiable. The emerging viewpoint of MUSes as the root causes of unsatisfiability has led MUSes to find applications in a wide variety of diagnostic approaches. Recent advances in identification and enumeration of MUSes have motivated researchers to discover applications that can benefit from rich information about the set of MUSes. One such extension is that of counting the number of MUSes. The current best approach for MUS counting is to employ a MUS enumeration algorithm, which often does not scale for the cases with a reasonably large number of MUSes. Motivated by the success of hashing-based techniques in the context of model counting, we design the first approximate MUS counting procedure with guarantees, called . Our approach avoids exhaustive MUS enumeration by combining the classical technique of universal hashing with advances in QBF solvers along with a novel usage of union and intersection of MUSes to achieve runtime efficiency. Our prototype implementation of is shown to scale to instances that were clearly beyond the realm of enumeration-based approaches.

show abstract

Proof-based coverage metrics for formal verification

Cited by 15 publications

References 20 publications

Merit and Blame Assignment with Kind 2

Merit and Blame Assignment with Kind 2

Specification Quality Metrics Based on Mutation and Inductive Incremental Model Checking

Approximate Counting of Minimal Unsatisfiable Subsets

Contact Info

Product

Resources

About