Learning invariants using decision trees and implication counterexamples

Garg, Pranav; Neider, Daniel; Madhusudan, P.; Roth, Dan

doi:10.1145/2837614.2837664

Cited by 128 publications

(135 citation statements)

References 36 publications

Supporting

Mentioning

135

Contrasting

Order By: Relevance

“…Algorithms performing only inductiveness checks can in fact be very sophisticated, traversing the domain of candidates in clever ways. This approach was formulated in the ICE learning framework for learning inductive invariants [Garg et al 2014[Garg et al , 2016 (later extended to general Constrained-Horn Clauses [Ezudheen et al 2018]), in which algorithms present new candidates based on positive, negative, and implication examples returned by a "teacher" in response to incorrect candidate invariants. 4 The main point is that such algorithms do not perform queries other than inductiveness, and choose the next candidate invariant based solely on the counterexamples to induction showing the previous candidates were unsuitable.…”

Section: Inference Using Rich Queriesmentioning

confidence: 99%

“…ICE. The ICE framework [Garg et al 2014[Garg et al , 2016 (later extended to general Constrained-Horn Clauses [Ezudheen et al 2018]), is a learning framework for inferring invariants from positive, negative and implication counterexamples. We now review the framework using the original terminology and notation; later in the paper we will use a related formulation that emphasizes the choice of candidates (in §7.1).…”

Section: Invariant Inference Algorithmsmentioning

confidence: 99%

“…The importance of implication counterexamples is that they allow implementing a teacher using a SAT/SMT solver without "guessing" what a counterexample to induction indicates [Garg et al 2014;Löding et al 2016]. Examples of ICE learning algorithms include Houdini [Flanagan and Leino 2001] and symbolic abstraction [Reps et al 2004;Thakur et al 2015], as well as designated algorithms [Garg et al 2014[Garg et al , 2016. Theoretically, the analysis of Garg et al [2014] focuses on strong convergence of the learner, namely, that the learner can always reach a correct concept, no matter how the teacher chooses to extend samples between rounds.…”

Section: Invariant Inference Algorithmsmentioning

confidence: 99%

See 2 more Smart Citations

Complexity and information in invariant inference

Feldman

Immerman

Sagiv

et al. 2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

This paper addresses the complexity of SAT-based invariant inference, a prominent approach to safety verification. We consider the problem of inferring an inductive invariant of polynomial length given a transition system and a safety property. We analyze the complexity of this problem in a black-box model, called the Hoare-query model, which is general enough to capture algorithms such as IC3/PDR and its variants. An algorithm in this model learns about the system's reachable states by querying the validity of Hoare triples.We show that in general an algorithm in the Hoare-query model requires an exponential number of queries. Our lower bound is information-theoretic and applies even to computationally unrestricted algorithms, showing that no choice of generalization from the partial information obtained in a polynomial number of Hoare queries can lead to an efficient invariant inference procedure in this class.We then show, for the first time, that by utilizing rich Hoare queries, as done in PDR, inference can be exponentially more efficient than approaches such as ICE learning, which only utilize inductiveness checks of candidates. We do so by constructing a class of transition systems for which a simple version of PDR with a single frame infers invariants in a polynomial number of queries, whereas every algorithm using only inductiveness checks and counterexamples requires an exponential number of queries.Our results also shed light on connections and differences with the classical theory of exact concept learning with queries, and imply that learning from counterexamples to induction is harder than classical exact learning from labeled examples. This demonstrates that the convergence rate of Counterexample-Guided Inductive Synthesis depends on the form of counterexamples.

show abstract

Section: Inference Using Rich Queriesmentioning

confidence: 99%

Section: Invariant Inference Algorithmsmentioning

confidence: 99%

Section: Invariant Inference Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation

Complexity and information in invariant inference

Feldman

Immerman

Sagiv

et al. 2019

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…A new class of black-box techniques based on learning has emerged in recent years to synthesize inductive invariants [Garg et al 2014[Garg et al , 2016. In this technique, there are two distinct agents, the Learner and the Teacher.…”

Section: Introductionmentioning

confidence: 99%

Horn-ICE learning for synthesizing invariants and contracts

Ezudheen¹,

Neider

D’Souza³

et al. 2018

Proc. ACM Program. Lang.

Self Cite

View full text Add to dashboard Cite

We design learning algorithms for synthesizing invariants using Horn implication counterexamples (Horn-ICE), extending the ICE learning model. In particular, we describe a decision tree learning algorithm that learns from non-linear Horn-ICE samples, works in polynomial time, and uses statistical heuristics to learn small trees that satisfy the samples. Since most verification proofs can be modeled using non-linear Horn clauses, Horn-ICE learning is a more robust technique to learn inductive annotations that prove programs correct. Our experiments show that an implementation of our algorithm is able to learn adequate inductive invariants and contracts efficiently for a variety of sequential and concurrent programs.

show abstract

“…In the short time that the formalism has been in public circulation, it has already performed well in its goal of facilitating research in synthesis while providing a basis for objective comparison of different algorithms. For example, the competition has provided important insights into the relative merits of different algorithms [3,2,7] which have been exploited to help develop and evaluate new algorithms [11,15,24,26,18,4,12]. Beyond synthesizer developers, there is a growing community of users that is coalescing around the formalism.…”

Section: Introductionmentioning

confidence: 99%

SyGuS-Comp 2016: Results and Analysis

Alur

Fisman

Singh

et al. 2016

Electron. Proc. Theor. Comput. Sci.

View full text Add to dashboard Cite

Syntax-Guided Synthesis (SyGuS) is the computational problem of finding an implementation f that meets both a semantic constraint given by a logical formula $\varphi$ in a background theory T, and a syntactic constraint given by a grammar G, which specifies the allowed set of candidate implementations. Such a synthesis problem can be formally defined in SyGuS-IF, a language that is built on top of SMT-LIB. The Syntax-Guided Synthesis Competition (SyGuS-Comp) is an effort to facilitate, bring together and accelerate research and development of efficient solvers for SyGuS by providing a platform for evaluating different synthesis techniques on a comprehensive set of benchmarks. In this year's competition we added a new track devoted to programming by examples. This track consisted of two categories, one using the theory of bit-vectors and one using the theory of strings. This paper presents and analyses the results of SyGuS-Comp'16.Comment: In Proceedings SYNT 2016, arXiv:1611.07178. arXiv admin note: text overlap with arXiv:1602.0117

show abstract

Learning invariants using decision trees and implication counterexamples

Cited by 128 publications

References 36 publications

Complexity and information in invariant inference

Complexity and information in invariant inference

Horn-ICE learning for synthesizing invariants and contracts

SyGuS-Comp 2016: Results and Analysis

Contact Info

Product

Resources

About