X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension

Abdou, Mostafa; Sas, Cezar; Aralikatte, Rahul; Augenstein, Isabelle; Søgaard, Anders

doi:10.18653/v1/d19-6130

Cited by 7 publications

(17 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The improvement is +1.04%, +0.89% and +1.47% in average F 1 score compared to XLM-15, XLM-R base and XLM-R large , respectively. We also evaluate on the less widely used crosslingual QA dataset X-WikiRE (Abdou et al, 2019) for which we observe similar result trends, and 0.55% improvement in terms of average F 1 score on zero-shot QA. More details can be found in the Appendix (Section A.1).…”

Section: Zero-and Few-shot Cross-lingual Nlimentioning

confidence: 58%

“…Few-shot learning methods have initially been introduced within the area of image classification (Vinyals et al, 2016;Ravi and Larochelle, 2017;Finn et al, 2017), but have recently also been applied to NLP tasks such as relation extraction (Han et al, 2018), text classification Rethmeier and Augenstein, 2020) and machine translation (Gu et al, 2018). Specifically, in NLP, these few-shot learning approaches include: (i) the transformation of the problem into a different task (e.g., relation extraction is transformed to question answering (Levy et al, 2017;Abdou et al, 2019)); or (ii) meta-learning (Andrychowicz et al, 2016;Finn et al, 2017).…”

Section: Related Workmentioning

confidence: 99%

“…In contrast to these tasks, which have seen much cross-lingual attention (Plank et al, 2016;Bjerva, 2017;Kementchedjhieva et al, 2018;de Lhoneux et al, 2018), there has been relatively little work on cross-lingual NLU, partly due to lack of benchmark datasets. Existing work has mainly been focused on NLI (Agic and Schluter, 2018;Conneau et al, 2018;Zhao et al, 2020), and to a lesser degree on RE (Faruqui and Kumar, 2015;Verga et al, 2016) and QA (Abdou et al, 2019;Lewis et al, 2020). Previous research generally reports that cross-lingual learning is challenging and that it is hard to beat a machine translation baseline (e.g., Conneau et al (2018)).…”

Section: Related Workmentioning

confidence: 99%

“…For example, a relation type such as author is transformed to at least one language question template (e.g., who is the author of x?, where x is an entity). Building on the work of Levy et al (2017), a new dataset (X-WikiRE) is introduced for multilingual QA-based relation extraction in five languages (i.e., English, French, Spanish, Italian and German) by Abdou et al (2019). Each instance in the dataset includes a question, a context, and an answer.…”

Section: A Appendicesmentioning

confidence: 99%

See 3 more Smart Citations

Zero-Shot Cross-Lingual Transfer with Meta Learning

Nooralahzadeh¹,

Bekoulis²,

Bjerva³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Learning what to share between tasks has become a topic of great importance, as strategic sharing of knowledge has been shown to improve downstream task performance. This is particularly important for multilingual applications, as most languages in the world are under-resourced. Here, we consider the setting of training models on multiple different languages at the same time, when little or no data is available for languages other than English. We show that this challenging setup can be approached using meta-learning: in addition to training a source language model, another model learns to select which training instances are the most beneficial to the first. We experiment using standard supervised, zero-shot cross-lingual, as well as fewshot cross-lingual settings for different natural language understanding tasks (natural language inference, question answering). Our extensive experimental setup demonstrates the consistent effectiveness of meta-learning for a total of 15 languages. We improve upon the state-of-the-art for zero-shot and few-shot NLI (on MultiNLI and XNLI) and QA (on the MLQA dataset). A comprehensive error analysis indicates that the correlation of typological features between languages can partly explain when parameter sharing learned via meta-learning is beneficial.

show abstract

Section: Zero-and Few-shot Cross-lingual Nlimentioning

confidence: 58%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: A Appendicesmentioning

confidence: 99%

See 2 more Smart Citations

Zero-Shot Cross-Lingual Transfer with Meta Learning

Nooralahzadeh¹,

Bekoulis²,

Bjerva³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…For the pretraining process, they collect pairs of English sentences based on the shared entities, annotated by an entity linking system. On the other hand, we propose a multilingual approach that utilizes Wikipedia and Wikidata, which are already available for many languages and have been successfully used for tasks such as multilingual question answering (Abdou et al, 2019) and named entity recognition (Nothman et al, 2013).…”

Section: Related Workmentioning

confidence: 99%

The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification

Köksal

Özgür

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Relation classification is one of the key topics in information extraction, which can be used to construct knowledge bases or to provide useful information for question answering. Current approaches for relation classification are mainly focused on the English language and require lots of training data with human annotations. Creating and annotating a large amount of training data for low-resource languages is impractical and expensive. To overcome this issue, we propose two cross-lingual relation classification models: a baseline model based on Multilingual BERT and a new multilingual pretraining setup, which significantly improves the baseline with distant supervision. For evaluation, we introduce a new public benchmark dataset for cross-lingual relation classification in English, French, German, Spanish, and Turkish, called RELX. We also provide the RELX-Distant dataset, which includes hundreds of thousands of sentences with relations from Wikipedia and Wikidata collected by distant supervision for these languages. Our code and data are available at: https://github.com/boun-tabi/RELX 1 English Translation: CD Laredo was founded in 1927 with the name "Sociedad Deportiva Charlestón".

show abstract

QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

2023

View full text Add to dashboard Cite

Alongside huge volumes of research on deep learning models in NLP in the recent years, there has been also much work on benchmark datasets needed to track modeling progress. Question answering and reading comprehension have been particularly prolific in this regard, with over 80 new datasets appearing in the past two years. This study is the largest survey of the field to date. We provide an overview of the various formats and domains of the current resources, highlighting the current lacunae for future work. We further discuss the current classifications of “skills” that question answering/reading comprehension systems are supposed to acquire, and propose a new taxonomy. The supplementary materials survey the current multilingual resources and monolingual resources for languages other than English, and we discuss the implications of over-focusing on English. The study is aimed at both practitioners looking for pointers to the wealth of existing data, and at researchers working on new resources.

show abstract

X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension

Cited by 7 publications

References 27 publications

Zero-Shot Cross-Lingual Transfer with Meta Learning

Zero-Shot Cross-Lingual Transfer with Meta Learning

The RELX Dataset and Matching the Multilingual Blanks for Cross-Lingual Relation Classification

QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

Contact Info

Product

Resources

About