Component-based synthesis for complex APIs

Abstract

Component-based approaches to program synthesis assemble programs from a database of existing components, such as methods provided by an API. In this paper, we present a novel type-directed algorithm for component-based synthesis. The key novelty of our approach is the use of a compact Petri-net representation to model relationships between methods in an API. Given a target method signature S, our approach performs reachability analysis on the underlying Petri-net model to identify sequences of method calls that could be used to synthesize an implementation of S. The programs synthesized by our algorithm are guaranteed to type check and pass all test cases provided by the user. We have implemented this approach in a tool called SYPET, and used it to successfully synthesize real-world programming tasks extracted from on-line forums and existing code repositories. We also compare SYPET with two state-of-the-art synthesis tools, namely INSYNTH and CODEHINT, and demonstrate that SYPET can synthesize more programs in less time. Finally, we compare our approach with an alternative solution based on hypergraphs and demonstrate its advantages.

FrAngel: component-based synthesis with control structures

“…Special-case similarity does not always hold. For example, the following program (a benchmark used by SyPet [Feng et al 2017b]) is not SCS:…”

“…Examples were drawn from existing unit tests if available, with some additional examples added to clarify functionality and expose special cases where needed. • SyPet: We also use the 30 tasks used in the evaluation of SyPet [Feng et al 2017b], none of which involve control structures. To match SyPet's search space as closely as possible, for these tasks we prevent FrAngel from using control structures, primitive operators, literals, and object fields.…”

“…Our experiments compare FrAngel to SyPet [Feng et al 2017b], an existing component-based synthesizer using a datastructure called a Petri net. We note that SyPet does not support control structures, but aside from SyPet, we are not aware of any other synthesizer that can generate an entire (multi-line) function implementation using arbitrary Java classes.…”

“…Synthesizing control structures. As noted in Section 1, many component-based synthesizers cannot handle control structures, especially loops [Feng et al 2017b;Gulwani et al 2011a;Jha et al 2010;Menon et al 2013]. Beyond component-based synthesis, some approaches in the wider program synthesis literature do handle loops [Bar-David and Taubenfeld 2003;Barthe et al 2013;Heule et al 2015;Qi et al 2012;Srivastava et al 2010], but many do not [Alur et al 2017;Balog et al 2017;Devlin et al 2017;Lau et al 2000;].…”

“…We created a benchmark suite including 90 tasks of widely varying difficulty, using a variety of libraries and control structure patterns. We also include the 30 tasks used to evaluate SyPet [Feng et al 2017b], the previous approach most similar to ours. While SyPet solves 28% of the 120 tasks within 30 minutes, FrAngel solves 94% within 30 minutes and 47% within 10 seconds.…”