Synthesizing transformations on hierarchically structured data

Abstract

This paper presents a new approach for synthesizing transformations on tree-structured data, such as Unix directories and XML documents. We consider a general abstraction for such data, called hierarchical data trees (HDTs) and present a novel example-driven synthesis algorithm for HDT transformations. Our central insight is to reduce the problem of synthesizing tree transformers to the synthesis of list transformations that are applied to the paths of the tree. The synthesis problem over lists is solved using a new algorithm that combines SMT solving and decision tree learning. We have implemented our technique in a system called HADES and show that HADES can automatically synthesize a variety of interesting transformations collected from online forums.

Inductive program synthesis over noisy data

“…In recent years there has been significant interest in learning programs from input-output examples. These techniques have been successfully used to synthesize programs for domains such as string and format transformations [10,18], data wrangling [7], data completion [20], and data structure manipulation [8,12,21]. Even though these efforts have been largely successful, they do not aspire to work with noisy data sets that may contain corrupted input-output examples.…”

Maximal multi-layer specification synthesis

“…As we mentioned earlier, most PBE synthesizers [5,13,16,30,31,33] perform program enumeration until they find a program that satisfies the input-output examples provided by the user. In order to perform program enumeration, we first need to represent the set of all possible programs up to a given depth.…”

“…Our technique is related to a line of work on programming-by-example (PBE) [5,13,16,30,31,33]. PBE has been widely applied to different domains such as string manipulation [5,16], data wrangling [13,31], and SQL queries [30,33]. Among these techniques, the Morpheus tool is directly related to the data wrangling client to which Mars is instantiated.…”

“…To mitigate this problem, in recent years, there has been significant interest in end-user program synthesis for data science, in which the goal is to automate tedious data analytics tasks from informal specifications, such as input-output examples [13,16] or natural language [27,32]. For instance, programming-by-example (PBE) has been used to automate tedious tasks such as string manipulations in Excel [16], data wrangling tasks on tabular and hierarchical data [13,31], and SQL queries [30]. Despite significant progress in PBE systems, expressing the user intent still remains a major challenge.…”

“…As a result, it is very challenging to derive the right mapping from the textual information to their corresponding function calls. Second, even if we have the right mapping, it is still unclear how to integrate this information into most existing PBE systems [5,16,30,31,33], which typically rely on their efficient search algorithms by leveraging the syntax or semantics of the input-output examples.…”