ICoq: Regression proof selection for large-scale verification projects

Çelik, Ahmet; Palmskog, Karl; Gligoric, Milos

doi:10.1109/ase.2017.8115630

Cited by 15 publications

(13 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regression proof selection (RPS) is the analogue of RTS for formal proofs, which, similarly to tests, can take a long time to check. The RPS technique implemented in the iCoq tool for Coq [12] uses hierarchical selection [11], where impacted files are used to locate impacted proofs to be checked.…”

Section: Regression Test Selection and Regression Proof Selectionmentioning

confidence: 99%

“…To ensure the adequacy of our formal model, we performed an empirical study using Chip. Specifically, we integrated Chip with Ekstazi [26], a tool for class-based regression test selection in Java, with iCoq [11], a tool for regression proof selection in Coq itself, and with Tup [54], a build system similar to make, replacing the existing components for change impact analysis in all these tools. We then compared the outcome and running time between the respective modified and original tool versions when applied to the revision histories of several open-source projects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Practical Machine-Checked Formalization of Change Impact Analysis

Palmskog

Çelik

Gligoric

2020

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

Change impact analysis techniques determine the components affected by a change to a software system, and are used as part of many program analysis techniques and tools, e.g., in regression test selection, build systems, and compilers. The correctness of such analyses usually depends both on domain-specific properties and change impact analysis, and is rarely established formally, which is detrimental to trustworthiness. We present a formalization of change impact analysis with machine-checked proofs of correctness in the Coq proof assistant. Our formal model factors out domain-specific concerns and captures system components and their interrelations in terms of dependency graphs. Using compositionality, we also capture hierarchical impact analysis formally for the first time, which, e.g., can capture when impacted files are used to locate impacted tests inside those files. We refined our verified impact analysis for performance, extracted it to efficient executable OCaml code, and integrated it with a regression test selection tool, one regression proof selection tool, and one build system, replacing their existing impact analyses. We then evaluated the resulting toolchains on several open source projects, and our results show that the toolchains run with only small differences compared to the original running time. We believe our formalization can provide a basis for formally proving domain-specific techniques using change impact analysis correct, and our verified code can be integrated with additional tools to increase their reliability.

show abstract

Section: Regression Test Selection and Regression Proof Selectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Practical Machine-Checked Formalization of Change Impact Analysis

Palmskog

Çelik

Gligoric

2020

Tools and Algorithms for the Construction and Analysis of Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…Proof development environments need to allow users to efficiently write, check, and share proofs (Faithfull et al, 2018); proof libraries need to allow easy search and seamless integration of results into local developments (Gauthier and Kaliszyk, 2015). Evolving projects face the possibility of previous proofs breaking due to seemingly unrelated changes, justifying design principles (Woos et al, 2016) as well as support for quick error detection (Celik et al, 2017) and repair .…”

Section: Challenges At Scalementioning

confidence: 99%

“…However, even with close conceptual ties between construction of programs and proofs, research in software engineering requires careful translation to the world of formal proofs. For example, proof engineers can benefit from regression testing techniques by considering lemmas and their proofs in place of tests, as in regression proving (Celik et al, 2017); yet, the standard metric used to prioritize regression tests-statement coverage-has no clear analogue for lemmas with complex conditions and quantification.…”

Section: Challenges At Scalementioning

confidence: 99%

“…This can negatively affect the productivity of proof engineers. Absent domainand context-specific knowledge, as in proof refactoring, the two main techniques to speed up regression proving are proof-checking parallelization (Wenzel, 2013b;Barras et al, 2013) and proof selection (Celik et al, 2017). Support for parallelization varies in degree and kind among proof assistants.…”

Section: Regression Provingmentioning

confidence: 99%

See 1 more Smart Citation

QED at Large: A Survey of Engineering of Formally Verified Software

Ringer

Palmskog

Sergey

et al. 2019

FNT in Programming Languages

Self Cite

View full text Add to dashboard Cite

Development of formal proofs of correctness of programs can increase actual and perceived reliability and facilitate better understanding of program specifications and their underlying assumptions. Tools supporting such development have been available for over 40 years, but have only recently seen wide practical use. Projects based on construction of machine-checked formal proofs are now reaching an unprecedented scale, comparable to large software projects, which leads to new challenges in proof development and maintenance. Despite its increasing importance, the field of proof engineering is seldom considered in its own right; related theories, techniques, and tools span many fields and venues. This survey of the literature presents a holistic understanding of proof engineering for program correctness, covering impact in practice, foundations, proof automation, proof organization, and practical proof development.

show abstract

Difference Verification with Conditions

Beyer

Jakobs

Lemberger

2020

Software Engineering and Formal Methods

View full text Add to dashboard Cite

Modern software-verification tools need to support development processes that involve frequent changes. Existing approaches for incremental verification hard-code specific verification techniques. Some of the approaches must be tightly intertwined with the development process. To solve this open problem, we present the concept of difference verification with conditions. Difference verification with conditions is independent from any specific verification technique and can be integrated in software projects at any time. It first applies a change analysis that detects which parts of a software were changed between revisions and encodes that information in a condition. Based on this condition, an off-the-shelf verifier is used to verify only those parts of the software that are influenced by the changes. As a proof of concept, we propose a simple, syntax-based change analysis and use difference verification with conditions with three off-the-shelf verifiers. An extensive evaluation shows the competitiveness of difference verification with conditions.

show abstract

ICoq: Regression proof selection for large-scale verification projects

Cited by 15 publications

References 38 publications

Practical Machine-Checked Formalization of Change Impact Analysis

Practical Machine-Checked Formalization of Change Impact Analysis

QED at Large: A Survey of Engineering of Formally Verified Software

Difference Verification with Conditions

Contact Info

Product

Resources

About