MetaVal: Witness Validation via Verification

Abstract

Witness validation is an important technique to increase trust in verification results, by making descriptions of error paths (violation witnesses) and important parts of the correctness proof (correctness witnesses) available in an exchangeable format. This way, the verification result can be validated independently from the verification in a second step. The problem is that there are unfortunately not many tools available for witness-based validation of verification results. We contribute to closing this gap with the approach of validation via verification, which is a way to automatically construct a set of validators from a set of existing verification engines. The idea is to take as input a specification, a program, and a verification witness, and produce a new specification and a transformed version of the original program such that the transformed program satisfies the new specification if the witness is useful to confirm the result of the verification. Then, an 'off-the-shelf' verifier can be used to validate the previously computed result (as witnessed by the verification witness) via an ordinary verification task. We have implemented our approach in the validator MetaVal, and it was successfully used in SV-COMP 2020 and confirmed 3 653 violation witnesses and 16 376 correctness witnesses. The results show that MetaVal improves the effectiveness (167 uniquely confirmed violation witnesses and 833 uniquely confirmed correctness witnesses) of the overall validation process, on a large benchmark set. All components and experimental data are publicly available.

Advances in Automatic Software Verification: SV-COMP 2020

“…All SV-COMP verifiers are required to justify the result (True or False) by producing a verification witness (except for those categories for which no witness validator is available). We used six independently developed witness-based result validators [19,20,21,25,66]. The majority of witnesses that the verifiers produced can be confirmed by the results-validation process.…”

“…A few categories were excluded from validation if the validators did not sufficiently support a certain kind of program or property. Two new validators participated in SV-COMP 2020: Nitwit [66] and MetaVal [25].…”

Verifying Security Vulnerabilities in Large Software Systems using Multi-Core k-Induction

“…(5) The violation type, i.e., NULL Pointer. Using "-w" script argument, it is possible to generate GraphML file [34] of each vulnerability found in order to use CPAchecker [17] to perform a counterexample validation.…”

“…1 C o u n t e r e x a m p l e : A counterexample validation was performed using existing tools to ensure that vulnerabilities we have found are real ones. The first tool we used was CPAchecker [17], which is a comprehensive tool for configurable software verification and contains a module to validate counterexamples [34], i.e., witness validation, and is used in SV-COMP to validate the results of the competition. Moreover, model checkers implement an option to output the counterexample as a GraphML file [34] and CPAchecker uses such file, the source-code, and a specification file, i.e., a definition of the type of property violation in the program.…”

“…In particular, we pre-process the input source-code files and then guide the model checker to explore specific locations in the state-space, e.g., functions, so that the usage of the computational resources is reduced and it can find real bugs in large software systems. Once a counterexample is found, we use existing approaches to check whether such violation is reproducible in the program or not [16], [17]. Our main contribution is to propose a novel approach for detecting and exploiting security vulnerabilities in large software systems.…”