Transform-Data-by-Example (TDE)

He, Yeye; Ganjam, Kris; Lee, Kukjin; Wang, Yue; Narasayya, Vivek; Chaudhuri, Surajit; Chu, Xu; Zheng, Yudian

doi:10.1145/3183713.3193539

Cited by 12 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FlashFill [6], for example, learns string transformation rules called domain-specific language (DSL) from input-output examples provided by users. PBE enables users to reformat or transform unstructured data without requiring any programming knowledge, and some data transformation systems using PBE have been proposed for this purpose [3,6,9,12,24]. FlashFill [6] synthesizes an algorithm for domain-specific language, L, from a small number of examples using version-space algebra [18] to succinctly represent a large set of programs in L. Although these techniques are useful for nonprogrammers, their use is not sufficiently widespread because of the lack of usability and confidence in the correctness of the synthesized programs [17].…”

Section: Figure 1: Example Of Data Transformation In Titanic Datasetmentioning

confidence: 99%

See 1 more Smart Citation

Data-centric disambiguation for data transformation with programming-by-example

Narita

Maudet

Lü

et al. 2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

Programming-by-example (PBE), can be a powerful tool to reduce manual work in repetitive data transformation tasks. However, few examples often leave ambiguity and may cause undesirable data transformation by the system. This ambiguity can be resolved by allowing the user to directly edit the synthesized programs; however, this is difficult for non-programmers. Here, we present a novel approach: data-centric disambiguation for data transformation, where users resolve the ambiguity in data transformation by examining and modifying the output rather than the program. The key idea is to focus on the given set of data the user wants to transform instead of pursuing the synthesized program's generality or completeness. Our system provides visualization and interaction methods that allow users to efficiently examine and fix the transformed outputs, which is much simpler than understanding and modifying the program itself. The user study suggests that our system can successfully help non-programmers to more easily and efficiently process data. CCS CONCEPTS• Human-centered computing → User interface programming.

show abstract

Section: Figure 1: Example Of Data Transformation In Titanic Datasetmentioning

confidence: 99%

“…FlashFill [6] and BlinkFill [24] construct directed acyclic graphs (DAG) using regular expressions to infer string transformation rules from input-output examples. Similar PBE-based systems that support data transformation also exist [9,12]. These systems do not require users to have prior programming skills.…”

Section: Related Workmentioning

confidence: 99%

Data-centric disambiguation for data transformation with programming-by-example

Narita

Maudet

Lü

et al. 2021

26th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

show abstract

“…Data transformation is a long-standing problem and a common step in data-pipelines. Significant progress has been made in this area, with recent work developing "by-pattern" [34] and "by-example" approaches [16,26,28,29,[31][32][33]43]. Research in this area have generated substantial impacts on commercial systems, with by-example features shipping in popular systems such as Microsoft Excel [26], Power BI [28], and Trifacta [7].…”

Section: Related Workmentioning

confidence: 99%

Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search

Yang,

He,

Chaudhuri

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent work has made significant progress in helping users to automate single data preparation steps, such as string-transformations and table-manipulation operators (e.g., Join, GroupBy, Pivot, etc.). We in this work propose to automate multiple such steps endto-end, by synthesizing complex data-pipelines with both stringtransformations and table-manipulation operators.We propose a novel by-target paradigm that allows users to easily specify the desired pipeline, which is a significant departure from the traditional by-example paradigm. Using by-target, users would provide input tables (e.g., csv or json files), and point us to a "target table" (e.g., an existing database table or BI dashboard) to demonstrate how the output from the desired pipeline would schematically "look like". While the problem is seemingly underspecified, our unique insight is that implicit table constraints such as FDs and keys can be exploited to significantly constrain the space to make the problem tractable. We develop an Auto-Pipeline system that learns to synthesize pipelines using reinforcementlearning and search. Experiments on large numbers of real pipelines crawled from GitHub suggest that Auto-Pipeline can successfully synthesize 60-70% of these complex pipelines (up to 10 steps) in 10-20 seconds on average.

show abstract

Effective entity matching with transformers

Suhara³

et al. 2023

The VLDB Journal

View full text Add to dashboard Cite

Transform-Data-by-Example (TDE)

Cited by 12 publications

References 8 publications

Data-centric disambiguation for data transformation with programming-by-example

Data-centric disambiguation for data transformation with programming-by-example

Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search

Effective entity matching with transformers

Contact Info

Product

Resources

About