DBPal: A Fully Pluggable NL2SQL Training Pipeline

Weir, Nathaniel; Utama, Prasetya Ajie; Galakatos, Alex; Crotty, Andrew; Ilkhechi, Amir; Ramaswamy, Shekar; Bhushan, Rohin; Geisler, Nadja; Hättasch, Benjamin; Eger, Steffen; Çetintemel, Uǧur; Binnig, Carsten

doi:10.1145/3318464.3380589

Cited by 28 publications

(16 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bootstrapping a Semantic Parser. One line of prior work on quickly bootstrapping a semantic parser has focused on creating synthetic training examples from a grammar developed by hand (Campagna et al, 2019;Weir et al, 2020;Marzoev et al, 2020;Campagna et al, 2020) or derived automatically from existing data (Jia and Liang, 2016;. Wang et al (2015) described an approach to bootstrapping that uses a grammar to generate canonical forms, which are paraphrased by crowdworkers to produce training data "overnight."…”

Section: Introductionmentioning

confidence: 99%

Constrained Language Models Yield Few-Shot Semantic Parsers

Shin¹,

Lin²,

Thomson³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natural language.To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount of data and very little code to convert into English-like representations, our blueprint for rapidly bootstrapping semantic parsers leads to surprisingly effective performance on multiple community tasks, greatly exceeding baseline methods also trained on the same limited data.

show abstract

Section: Introductionmentioning

confidence: 99%

Constrained Language Models Yield Few-Shot Semantic Parsers

Shin¹,

Lin²,

Thomson³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

show abstract

“…Salvatore et al (2019) focus on textual entailment and probes models with synthetic examples. In semantic parsing Wang et al (2015); Iyer et al (2017); Weir et al (2020) use templates to augment the training data for text-to-SQL tasks and Geva et al (2020) do so to improve numerical reasoning, as do on tabular data. They also create minimal contrastive examples (Kaushik et al, 2020;Gardner et al, 2020) by automatically swapping entities in the statements by plausible alternatives that exists elsewhere in the table.…”

Section: Synthetic Datamentioning

confidence: 99%

TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-training

Müller¹,

Eisenschlos²,

Krichene³

2021

Preprint

View full text Add to dashboard Cite

We present the TAPAS contribution to the Shared Task on Statement Verification and Evidence Finding with Tables (SemEval 2021 Task 9, Wang et al. (2021)). SEMTABFACT Task A is a classification task of recognising if a statement is entailed, neutral or refuted by the content of a given table. We adopt the binary TAPAS model of Eisenschlos et al. (2020) to this task. We learn two binary classification models: A first model to predict if a statement is neutral or non-neutral and a second one to predict if it is entailed or refuted. As the shared task training set contains only entailed or refuted examples, we generate artificial neutral examples to train the first model. Both models are pre-trained using a MASKLM objective, intermediate counter-factual and synthetic data (Eisenschlos et al., 2020) and TABFACT (Chen et al., 2020), a large table entailment dataset. We find that the artificial neutral examples are somewhat effective at training the first model, achieving 68.03 test F1 versus the 60.47 of a majority baseline. For the second stage, we find that the pre-training on the intermediate data and TABFACT improves the results over MASKLM pre-training (68.03 vs 57.01).

show abstract

“…Both works rely on manual paraphrases and hand-tuned annotations on each database attribute. Training with synthetic data has also been explored to complement existed dataset (Weir et al, 2020) and in the few-shot setting .…”

Section: Related Workmentioning

confidence: 99%

AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data

Xu¹,

J.²,

Campagna³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose AutoQA, a methodology and toolkit to generate semantic parsers that answer questions on databases, with no manual effort. Given a database schema and its data, AutoQA automatically generates a large set of high-quality questions for training that covers different database operations. It uses automatic paraphrasing combined with templatebased parsing to find alternative expressions of an attribute in different parts of speech. It also uses a novel filtered auto-paraphraser to generate correct paraphrases of entire sentences.We apply AutoQA to the Schema2QA dataset and obtain an average logical form accuracy of 62.9% when tested on natural questions, which is only 6.4% lower than a model trained with expert natural language annotations and paraphrase data collected from crowdworkers. To demonstrate the generality of AutoQA, we also apply it to the Overnight dataset. AutoQA achieves 69.8% answer accuracy, 16.4% higher than the state-of-the-art zero-shot models and only 5.2% lower than the same model trained with human data.

show abstract

DBPal: A Fully Pluggable NL2SQL Training Pipeline

Cited by 28 publications

References 30 publications

Constrained Language Models Yield Few-Shot Semantic Parsers

Constrained Language Models Yield Few-Shot Semantic Parsers

TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-training

AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Data

Contact Info

Product

Resources

About