RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

Wang, Bailin; Shin, Richard; Liu, Xiaodong; Polozov, Oleksandr; Richardson, Matthew

doi:10.18653/v1/2020.acl-main.677

Cited by 283 publications

(352 citation statements)

References 21 publications

(26 reference statements)

Supporting

Mentioning

275

Contrasting

Order By: Relevance

“…Following Wang et al (2019), Equation 1describes attention scores between Query (Q) and Key (K), z h i is the h th attention head, applying scores s…”

Section: Neural Semantic Parsingmentioning

confidence: 99%

Bootstrapping a Crosslingual Semantic Parser

Sherborne

Lapata

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Recent progress in semantic parsing scarcely considers languages other than English but professional translation can be prohibitively expensive. We adapt a semantic parser trained on a single language, such as English, to new languages and multiple domains with minimal annotation. We query if machine translation is an adequate substitute for training data, and extend this to investigate bootstrapping using joint training with English, paraphrasing, and multilingual pre-trained models. We develop a Transformer-based parser combining paraphrases by ensembling attention over multiple encoders and present new versions of ATIS and Overnight in German and Chinese for evaluation. Experimental results indicate that MT can approximate training data in a new language for accurate parsing when augmented with paraphrasing through multiple MT engines. Considering when MT is inadequate, we also find that using our approach achieves parsing accuracy within 2% of complete translation using only 50% of training data. 1

show abstract

“…Following Wang et al (2019), Equation 1describes attention scores between Query (Q) and Key (K), z h i is the h th attention head, applying scores s…”

Section: Neural Semantic Parsingmentioning

confidence: 99%

Bootstrapping a Crosslingual Semantic Parser

Sherborne

Lapata

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

show abstract

“…Early systems used rule-based, pattern-based or grammar-based approaches to translate from natural language to SQL [5,7]. The introduction of the Spider leaderboard in 2018 has triggered a significant interest of several research groups to tackle the problem with machine learning approaches, in particular with advanced neural networks [16][17][18][19]. However, most of these systems provide solutions for translating from natural language to SQL rather than to SPARQL -which is the standardized query language for RDF graph databases.…”

Section: Related Workmentioning

confidence: 99%

Querying knowledge graphs in natural language

Liang¹,

Stockinger

Farias

et al. 2021

J Big Data

View full text Add to dashboard Cite

Knowledge graphs are a powerful concept for querying large amounts of data. These knowledge graphs are typically enormous and are often not easily accessible to end-users because they require specialized knowledge in query languages such as SPARQL. Moreover, end-users need a deep understanding of the structure of the underlying data models often based on the Resource Description Framework (RDF). This drawback has led to the development of Question-Answering (QA) systems that enable end-users to express their information needs in natural language. While existing systems simplify user access, there is still room for improvement in the accuracy of these systems. In this paper we propose a new QA system for translating natural language questions into SPARQL queries. The key idea is to break up the translation process into 5 smaller, more manageable sub-tasks and use ensemble machine learning methods as well as Tree-LSTM-based neural network models to automatically learn and translate a natural language question into a SPARQL query. The performance of our proposed QA system is empirically evaluated using the two renowned benchmarks-the 7th Question Answering over Linked Data Challenge (QALD-7) and the Large-Scale Complex Question Answering Dataset (LC-QuAD). Experimental results show that our QA system outperforms the state-of-art systems by 15% on the QALD-7 dataset and by 48% on the LC-QuAD dataset, respectively. In addition, we make our source code available.

show abstract

“…We obtain the development set model-predicted queries from 21 submissions. 5 They include models from Guo et al (2019); Bogin et al (2019b); Choi et al (2020); Wang et al (2020). 6 These models capture a broad diversity of network architectures, decoding strategies, and pre-traning methods, with accuracy ranging from below 40% to above 70%.…”

Section: Model-predicted Queriesmentioning

confidence: 99%

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

Zhong¹,

Yu²,

Klein³

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models. Our method distills a small test suite of databases that achieves high code coverage for the gold query from a large number of randomly generated databases. At evaluation time, it computes the denotation accuracy of the predicted queries on the distilled test suite, hence calculating a tight upper-bound for semantic accuracy efficiently. We use our proposed method to evaluate 21 models submitted to the Spider leader board and manually verify that our method is always correct on 100 examples. In contrast, the current Spider metric leads to a 2.5% false negative rate on average and 8.1% in the worst case, indicating that test suite accuracy is needed. Our implementation, along with distilled test suites for eleven Textto-SQL datasets, is publicly available. 11 Metric implementation and test suites available here, for datasets:

show abstract

RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

Cited by 283 publications

References 21 publications

Bootstrapping a Crosslingual Semantic Parser

Bootstrapping a Crosslingual Semantic Parser

Querying knowledge graphs in natural language

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

Contact Info

Product

Resources

About