Zero-shot Text-to-SQL Learning with Auxiliary Task

Abstract

Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task. However, little work has paid attention to how these models generalize to realistic unseen data, which naturally raises a question: does this impressive performance signify a perfect generalization model, or are there still some limitations?In this paper, we first diagnose the bottleneck of text-to-SQL task by providing a new testbed, in which we observe that existing models present poor generalization ability on rarely-seen data. The above analysis encourages us to design a simple but effective auxiliary task, which serves as a supportive model as well as a regularization term to the generation task to increase the models generalization. Experimentally, We evaluate our models on a large text-to-SQL dataset WikiSQL. Compared to a strong baseline coarse-to-fine model, our models improve over the baseline by more than 3% absolute in accuracy on the whole dataset. More interestingly, on a zero-shot subset test of WikiSQL, our models achieve 5% absolute accuracy gain over the baseline, clearly demonstrating its superior generalizability.

Few-Shot Semantic Parsing for New Predicates

“…To address this issue, one line of research is to augment existing datasets with automatically generated data (Su and Yan, 2017;Jia and Liang, 2016;Cai and Yates, 2013). Another line of research is to exploit available resources, such as knowledge bases (Krishnamurthy et al, 2017;Herzig and Berant, 2018;Chang et al, 2019;Lee, 2019;Zhang et al, 2019a;Guo et al, 2019;Wang et al, 2019), semantic features in different domains (Dadashkarimi et al, 2018;Li et al, 2020), or unlabeled data Kočiskỳ et al, 2016;Sun et al, 2019). Those works are orthogonal to our setting because our approach aims to efficiently exploit a handful of labeled data of new predicates, which are not limited to the ones in knowledge bases.…”

Hybrid Ranking Network for Text-to-SQL

“…The encoder encodes information from both the NL question and the table schema into some hidden representation. Some of them encode an NL question with the full table schema, e.g., concatenating the NL question with all the column names (Zhong et al, 2017;Dong and Lapata, 2018;Chang et al, 2019;Hwang et al, 2019;, while others encode an NL question with each column separately (Xu et al, 2017;Yu et al, 2018;Hwang et al, 2019). There are also some work do both at different layers (Hwang et al, 2019;.…”