The Power of Prompt Tuning for Low-Resource Semantic Parsing

Schucher, Nathan; Reddy, Siva; Vries, Harm de

doi:10.48550/arxiv.2110.08525

Cited by 5 publications

(7 citation statements)

References 16 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One is tuning-free prompting, for example, Petroni et al (2019); Shin et al (2020) used a fill-in-the-blank paradigm, while Brown et al ( 2020); Shin et al (2021) used "few-shot" prompts that included several examples of inputs followed by target outputs, with the actual task input appended at the end. One is Fixed-LM Prompt Tuning, as used by Li and Liang (2021); Schucher et al (2021); Qin and Eisner (2021); Liu et al (2021b), which requires training less parameters compared with tuning the whole model. Another is Fixed-prompt LM Tuning, which is similar to our setting.…”

Section: Compositional Generalization In Semanticmentioning

confidence: 99%

See 1 more Smart Citation

SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Yang¹,

Jiang²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent research showed promising results on combining pretrained language models (LMs) with canonical utterance for few-shot semantic parsing. The canonical utterance is often lengthy and complex due to the compositional structure of formal languages. Learning to generate such canonical utterance requires significant amount of data to reach high performance. Fine-tuning with only few-shot samples, the LMs can easily forget pretrained knowledge, overfit spurious biases, and suffer from compositionally out-of-distribution generalization errors. To tackle these issues, we propose a novel few-shot semantic parsing method -SEQZERO. SEQZERO decomposes the problem into a sequence of sub-problems, which correspond to the sub-clauses of the formal language. Based on the decomposition, the LMs only need to generate short answers using prompts for predicting sub-clauses. Thus, SEQZERO avoids generating a long canonical utterance at once. Moreover, SEQZERO employs not only a few-shot model but also a zero-shot model to alleviate the overfitting. In particular, SEQZERO brings out the merits from both models via ensemble equipped with our proposed constrained rescaling. SEQZERO achieves SOTA performance of BART-based models on GeoQuery and EcommerceQuery, which are two few-shot datasets with compositional data split. 1

show abstract

Section: Compositional Generalization In Semanticmentioning

confidence: 99%

“…Considering the difference between natural and formal language, adapting LMs to semantic parsing is non-trivial. Prior works typically first finetune the LMs to generate canonical utterance, which is then transformed into the final formal language through grammars (Shin et al, 2021;Schucher et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Yang¹,

Jiang²,

Yin³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Nevertheless, recent work (Shin et al, 2021;Wu et al, 2021;Yin et al, 2021;Schucher et al, 2021) has demonstrated decent performance given just a small seed dataset D, by combining pretrained language models with constrained decoding. For example, Shin et al (2021) use only 300 labeled examples in the complex SMCalFlow dialogue domain (Andreas et al, 2020).…”

Section: Related Workmentioning

confidence: 99%

Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation

Yang¹,

Deng²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

We introduce a novel setup for low-resource task-oriented semantic parsing which incorporates several constraints that may arise in real-world scenarios: (1) lack of similar datasets/models from a related domain, (2) inability to sample useful logical forms directly from a grammar, and (3) privacy requirements for unlabeled natural utterances. Our goal is to improve a low-resource semantic parser using utterances collected through user interactions. In this highly challenging but realistic setting, we investigate data augmentation approaches involving generating a set of structured canonical utterances corresponding to logical forms, before simulating corresponding natural language and filtering the resulting pairs. We find that such approaches are effective despite our restrictive setup: in a lowresource setting on the complex SMCalFlow calendaring dataset (Andreas et al., 2020), we observe 33% relative improvement over a nondata-augmented baseline in top-1 match.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Yang¹,

Jiang²,

Yin³

et al. 2022

Findings of the Association for Computational Linguistics: NAACL 2022

View full text Add to dashboard Cite

Recent research showed promising results on combining pretrained language models (LMs) with canonical utterance for few-shot semantic parsing. The canonical utterance is often lengthy and complex due to the compositional structure of formal languages. Learning to generate such canonical utterance requires significant amount of data to reach high performance. Fine-tuning with only few-shot samples, the LMs can easily forget pretrained knowledge, overfit spurious biases, and suffer from compositionally out-of-distribution generalization errors. To tackle these issues, we propose a novel few-shot semantic parsing method -SEQZERO. SEQZERO decomposes the problem into a sequence of sub-problems, which correspond to the sub-clauses of the formal language. Based on the decomposition, the LMs only need to generate short answers using prompts for predicting sub-clauses. Thus, SEQZERO avoids generating a long canonical utterance at once. Moreover, SEQZERO employs not only a fewshot model but also a zero-shot model to alleviate the overfitting. In particular, SEQZERO brings out the merits from both models via ensemble equipped with our proposed constrained rescaling. SEQZERO achieves SOTA performance of BART-based models on GeoQuery and EcommerceQuery, which are two few-shot datasets with compositional data split. 1

show abstract

The Power of Prompt Tuning for Low-Resource Semantic Parsing

Cited by 5 publications

References 16 publications

SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation

SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models

Contact Info

Product

Resources

About