Programming with enumerable sets of structures

Kuraj, Ivan; Kunčak, Viktor; Jackson, Daniel

doi:10.1145/2814270.2814323

Cited by 21 publications

(21 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We expose a new mechanism for enumeration that allows the programmer to define what it means for structures to be different. This is similar to what SciFe [Kuraj et al 2015] accomplishes, except we can express our enumerators to any SMT solver. This means that we can use established solvers, abstract away reasoning about data, and support constraints on strings (which most similar tools do not support).…”

Section: Our Constraint Problem Contrastedmentioning

confidence: 77%

A solver-aided language for test input generation

Ringer

Grossman

Schwartz-Narbonne

et al. 2017

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

Developing a small but useful set of inputs for tests is challenging. We show that a domain-specific language backed by a constraint solver can help the programmer with this process. The solver can generate a set of test inputs and guarantee that each input is different from other inputs in a way that is useful for testing. This paper presents Iorek: a tool that empowers the programmer with the ability to express to any SMT solver what it means for inputs to be different. The core of Iorek is a rich language for constraining the set of inputs, which includes a novel bounded enumeration mechanism that makes it easy to define and encode a flexible notion of difference over a recursive structure. We demonstrate the flexibility of this mechanism for generating strings. We use Iorek to test real services and find that it is effective at finding bugs. We also build Iorek into a random testing tool and show that it increases coverage. CCS Concepts: • Software and its engineering → Domain specific languages; Software testing and debugging; Constraints;

show abstract

Section: Our Constraint Problem Contrastedmentioning

confidence: 77%

A solver-aided language for test input generation

Ringer

Grossman

Schwartz-Narbonne

et al. 2017

Proc. ACM Program. Lang.

View full text Add to dashboard Cite

show abstract

“…It also best exploits the counterexample sharing optimization described above, as smaller sketches quickly generate a set of counterexamples that later local searches can use. We found the Cantor and Szudzik orders [19,38] to be particularly effective.…”

Section: Methodsmentioning

confidence: 88%

“…As in classic CEGIS, an invalid solution leads to a counterexample, which is added to Z (lines [12][13][14]. If there is no model for the problem, the search terminates (lines [19][20]. When the algorithm satisfies all the constraints it has received so far, it blocks until it receives new constraints.…”

Section: Local Searchesmentioning

confidence: 99%

Optimizing synthesis with metasketches

et al. 2016

View full text Add to dashboard Cite

Many advanced programming tools---for both end-users and expert developers---rely on program synthesis to automatically generate implementations from high-level specifications. These tools often need to employ tricky, custom-built synthesis algorithms because they require synthesized programs to be not only correct, but also optimal with respect to a desired cost metric, such as program size. Finding these optimal solutions efficiently requires domain-specific search strategies, but existing synthesizers hard-code the strategy, making them difficult to reuse. This paper presents metasketches, a general framework for specifying and solving optimal synthesis problems. metasketches make the search strategy a part of the problem definition by specifying a fragmentation of the search space into an ordered set of classic sketches. We provide two cooperating search algorithms to effectively solve metasketches. A global optimizing search coordinates the activities of local searches, informing them of the costs of potentially-optimal solutions as they explore different regions of the candidate space in parallel. The local searches execute an incremental form of counterexample-guided inductive synthesis to incorporate information sent from the global search. We present Synapse, an implementation of these algorithms, and show that it effectively solves optimal synthesis problems with a variety of different cost functions. In addition, metasketches can be used to accelerate classic (non-optimal) synthesis by explicitly controlling the search strategy, and we show that Synapse solves classic synthesis problems that state-of-the-art tools cannot.

show abstract

“…Two popular techniques are declarative enumeration and imperative enumeration. In particular, declarative approaches leverage any given invariant to search for valid inputs [7,19,28,50], and the imperative approaches directly construct the inputs based on more prescriptive specifications [12,31,49,56]. In program synthesis, there have been studies on inductive functional programming systems for exhaustively synthesizing small programs [8,[25][26][27].…”

Section: Related Workmentioning

confidence: 99%

Skeletal program enumeration for rigorous compiler testing

Zhang

Sun

2017

Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation

View full text Add to dashboard Cite

A program can be viewed as a syntactic structure P (syntactic skeleton) parameterized by a collection of identifiers V (variable names). This paper introduces the skeletal program enumeration (SPE) problem: Given a syntactic skeleton P and a set of variables V , enumerate a set of programs P exhibiting all possible variable usage patterns within P. It proposes an effective realization of SPE for systematic, rigorous compiler testing by leveraging three important observations: (1) Programs with different variable usage patterns exhibit diverse control-and data-dependence, and help exploit different compiler optimizations; (2) most real compiler bugs were revealed by small tests (i.e., small-sized P) -this "small-scope" observation opens up SPE for practical compiler validation; and (3) SPE is exhaustive w.r.t. a given syntactic skeleton and variable set, offering a level of guarantee absent from all existing compiler testing techniques.The key challenge of SPE is how to eliminate the enormous amount of equivalent programs w.r.t. α-conversion. Our main technical contribution is a novel algorithm for computing the canonical (and smallest) set of all non-α-equivalent programs. To demonstrate its practical utility, we have applied the SPE technique to test C/C++ compilers using syntactic skeletons derived from their own regression test-suites. Our evaluation results are extremely encouraging. In less than six months, our approach has led to 217 confirmed GCC/Clang bug reports, 119 of which have already been fixed, and the majority are long latent despite extensive prior testing efforts. Our SPE algorithm also provides six orders of magnitude reduction. Moreover, in three weeks, our technique has found 29 CompCert crashing bugs and 42 bugs in two Scala optimizing compilers. These results demonstrate our SPE technique's generality and further illustrate its effectiveness.

show abstract

Programming with enumerable sets of structures

Cited by 21 publications

References 49 publications

A solver-aided language for test input generation

A solver-aided language for test input generation

Optimizing synthesis with metasketches

Skeletal program enumeration for rigorous compiler testing

Contact Info

Product

Resources

About