Zero-Shot Text-to-SQL Learning with Auxiliary Task

Abstract

Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task. However, little work has paid attention to how these models generalize to realistic unseen data, which naturally raises a question: does this impressive performance signify a perfect generalization model, or are there still some limitations?In this paper, we first diagnose the bottleneck of the text-to-SQL task by providing a new testbed, in which we observe that existing models present poor generalization ability on rarely-seen data. The above analysis encourages us to design a simple but effective auxiliary task, which serves as a supportive model as well as a regularization term to the generation task to increase the models' generalization. Experimentally, We evaluate our models on a large text-to-SQL dataset WikiSQL. Compared to a strong baseline coarse-to-fine model, our models improve over the baseline by more than 3% absolute in accuracy on the whole dataset. More interestingly, on a zero-shot subset test of WikiSQL, our models achieve 5% absolute accuracy gain over the baseline, clearly demonstrating its superior generalizability.

Few-Shot Semantic Parsing for New Predicates

“…To address this issue, one line of research is to augment existing datasets with automatically generated data (Su and Yan, 2017;Jia and Liang, 2016;Cai and Yates, 2013). Another line of research is to exploit available resources, such as knowledge bases (Krishnamurthy et al, 2017;Herzig and Berant, 2018;Chang et al, 2019;Lee, 2019;Zhang et al, 2019a;Guo et al, 2019;Wang et al, 2019), semantic features in different domains (Dadashkarimi et al, 2018;Li et al, 2020), or unlabeled data Kočiskỳ et al, 2016;Sun et al, 2019). Those works are orthogonal to our setting because our approach aims to efficiently exploit a handful of labeled data of new predicates, which are not limited to the ones in knowledge bases.…”

SQL Generation from Natural Language: A Sequence-to-Sequence Model Powered by the Transformers Architecture and Association Rules

“…In this case, information from both the NL and table schema is encoded into a hidden representation by the encoder. Some of those works encode the question with each column name separately (Xu et al, 2017;Yu et al, 2018;Hwang et al, 2019) and other choose encoding the concatenation of the question with columns name (Zhong et al, 2017;Dong and Lapata, 2018;Chang et al, 2020;Hwang et al, 2019;He et al, 2019).…”

Semi-Automatic Construction of Text-to-SQL Data for Domain Transfer

“…Text-to-SQL Text-to-SQL as a semantic parsing task, has attracted increasing interest, where multiple large-scale datasets have been released. Zhong et al (2017) created a large single-table Text-to-SQL dataset, WikiSQL, from Wikipedia entries, upon which many semantic parsers have been trained, achieving high accuracies surpassing 80% (Chang et al, 2020;Lyu et al, 2020;Hwang et al, 2019;He et al, 2019). Yu et al (2018) proposed SPIDER, another large-scale text-to-SQL dataset with multi-table databases, much wider grammar coverage, and more complex queries.…”