AutoPandas: neural-backed generators for program synthesis

Abstract

Developers nowadays have to contend with a growing number of APIs. While in the long-term they are very useful to developers, many modern APIs have an incredibly steep learning curve, due to their hundreds of functions handling many arguments, obscure documentation, and frequently changing semantics. For APIs that perform data transformations, novices can often provide an I/O example demonstrating the desired transformation, but may be stuck on how to translate it to the API. A programming-by-example synthesis engine that takes such I/O examples and directly produces programs in the target API could help such novices. Such an engine presents unique challenges due to the breadth of real-world APIs, and the often-complex constraints over function arguments. We present a generator-based synthesis approach to contend with these problems. This approach uses a program candidate generator, which encodes basic constraints on the space of programs. We introduce neural-backed operators which can be seamlessly integrated into the program generator. To improve the efficiency of the search, we simply use these operators at non-deterministic decision points, instead of relying on domain-specific heuristics. We implement this technique for the Python pandas library in AutoPandas. AutoPandas supports 119 pandas dataframe transformation functions. We evaluate AutoPandas on 26 real-world benchmarks and find it solves 17 of them. CCS Concepts: • Software and its engineering → Programming by example; Automatic programming.

avatar—Automated Feature Wrangling for Machine Learning

“…Auto-Suggest recommends data preparation steps, such as pivot and join, for raw tables [17]. Foofah [6] and AutoPandas [1] learn full transformation programs if an output example can be given. A second type of wrangling is concerned with transforming and normalizing individual columns based on examples provided by a user [3,5].…”

HISyn: human learning-inspired natural language programming

“…HISyn combines a parent path with the child path by adding an edge directed from the leave of the parent path to the root of the child path. In Figure 3(f), path 4 and path 5 are connected to prefix tree (2,3), and prefix tree (2,3) is connected to path 1.…”

“…The rule-based approach had shown some success in the early stage of the field development (e.g., Smartsynth [25]), but have gradually lost attractions due to the lack of robustness and the difficulties in generalizing across domains. The data-driven approach has dominated recent efforts, represented by the adoption of deep learning to map NL queries to code via various neural networks (e.g., [2,14,29,38,39]). Although this approach has shown more promise than the previous rule-driven approach, its requirement of large numbers of labeled examples hinders its adoptions, especially for domains where labeled examples are scarce.…”

Program Synthesis Using Deduction-Guided Reinforcement Learning

“…For example, many recent synthesis techniques use lightweight program analysis or logical reasoning to significantly prune the search space [18,19,39,53]. On the other hand, several recent approaches utilize a statistical model (trained off-line) to bias the search towards programs that are more likely to satisfy the specification [2,4,7,19]. While both deductive and statistical reasoning have been shown to dramatically improve synthesis efficiency, a key limitation of existing approaches is that they do not tightly combine these two modes of reasoning.…”