Constrained Language Models Yield Few-Shot Semantic Parsers

Shin, Richard; Lin, Christopher H.; Thomson, Sam; Chen, Charles; Roy, Subhro; Platanios, Emmanouil Antonios; Pauls, Adam; Klein, Dan; Eisner, Jason; Durme, Benjamin Van

doi:10.48550/arxiv.2104.08768

Cited by 10 publications

(14 citation statements)

References 51 publications

(42 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The parse of each utterance is produced via few-shot prompting (Brown et al, 2020): the utterance is added to the end of the prompt, and the subsequent GPT-3 generation is interpreted as the target parse. We found that this simple parsing technique works well and could easily be applied to other parsing-based tasks, as in Shin et al (2021). The parsing prompts are reproduced in full in the Appendix.…”

Section: Integrating System 1 and Systemmentioning

confidence: 87%

Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning

Nye¹,

Tessler²,

Tenenbaum³

et al. 2021

Preprint

View full text Add to dashboard Cite

Human reasoning can often be understood as an interplay between two systems: the intuitive and associative ("System 1") and the deliberative and logical ("System 2"). Neural sequence models-which have been increasingly successful at performing complex, structured tasks-exhibit the advantages and failure modes of System 1: they are fast and learn patterns from data, but are often inconsistent and incoherent. In this work, we seek a lightweight, training-free means of improving existing System 1-like sequence models by adding System 2-inspired logical reasoning. We explore several variations on this theme in which candidate generations from a neural sequence model are examined for logical consistency by a symbolic reasoning module, which can either accept or reject the generations. Our approach uses neural inference to mediate between the neural System 1 and the logical System 2. Results in robust story generation and grounded instruction-following show that this approach can increase the coherence and accuracy of neurally-based generations.

show abstract

Section: Integrating System 1 and Systemmentioning

confidence: 87%

Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning

Nye¹,

Tessler²,

Tenenbaum³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Directly learning such a language model is challenging as the difference between the formal language and natural language is huge. To bridge the gap, Berant and Liang (2014); Shin et al (2021) propose Semantic Parsing via Paraphrasing (SPP) -a two-stage framework. In the first stage, they paraphrase u to its canonical utterance c using a paraphrasing language model:…”

Section: Preliminarymentioning

confidence: 99%

“…Considering the difference between natural and formal language, adapting LMs to semantic parsing is non-trivial. Prior works typically first finetune the LMs to generate canonical utterance, which is then transformed into the final formal language through grammars (Shin et al, 2021;Schucher et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Yang¹,

Jiang²,

Yin³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

Recent research showed promising results on combining pretrained language models (LMs) with canonical utterance for few-shot semantic parsing. The canonical utterance is often lengthy and complex due to the compositional structure of formal languages. Learning to generate such canonical utterance requires significant amount of data to reach high performance. Fine-tuning with only few-shot samples, the LMs can easily forget pretrained knowledge, overfit spurious biases, and suffer from compositionally out-of-distribution generalization errors. To tackle these issues, we propose a novel few-shot semantic parsing method -SEQZERO. SEQZERO decomposes the problem into a sequence of sub-problems, which correspond to the sub-clauses of the formal language. Based on the decomposition, the LMs only need to generate short answers using prompts for predicting sub-clauses. Thus, SEQZERO avoids generating a long canonical utterance at once. Moreover, SEQZERO employs not only a fewshot model but also a zero-shot model to alleviate the overfitting. In particular, SEQZERO brings out the merits from both models via ensemble equipped with our proposed constrained rescaling. SEQZERO achieves SOTA performance of BART-based models on GeoQuery and EcommerceQuery, which are two few-shot datasets with compositional data split. 1

show abstract

“…As with other NLP tasks, SOTA results in semantic parsing, including task-oriented parsing, have been obtained by fine-tuning large pretrained language models [6,14,17], particularly pretrained encoder-decoder architectures such as BART [12]. Shin et al [18] leverages the PLM's ability to generate natural language by reframing semantic parsing as a paraphrasing task. They use the PLM to map utterances into a canonical (but highly restricted) naturallanguage representation that can then be automatically mapped to the final output target.…”

Section: Related Workmentioning

confidence: 99%

Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models

Sun,

Khan,

Mesnards

et al. 2022

Preprint

View full text Add to dashboard Cite

Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attained by finetuning a large pretrained language model (PLM). While effective, this approach is inefficient in the presence of multiple downstream tasks, as a new set of values for all parameters of the PLM needs to be stored for each task separately. Recent work has explored methods for adapting PLMs to downstream tasks while keeping most (or all) of their parameters frozen. We examine two such promising techniques, prefix tuning and bias-term tuning, specifically on semantic parsing. We compare them against each other on two different semantic parsing datasets, and we also compare them against full and partial fine-tuning, both in few-shot and conventional data settings. While prefix tuning is shown to do poorly for semantic parsing tasks off the shelf, we modify it by adding special token embeddings, which results in very strong performance without compromising parameter savings. CCS CONCEPTS• Computing methodologies → Natural language processing; Neural networks.

show abstract

Constrained Language Models Yield Few-Shot Semantic Parsers

Cited by 10 publications

References 51 publications

Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning

Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning

SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models

Contact Info

Product

Resources

About