A Multilingual Reading Comprehension System for more than 100 Languages

Abstract

This paper presents M-GAAMA, a Multilingual Question Answering architecture and demo system. This is the first multilingual machine reading comprehension (MRC) demo which is able to answer questions in over 100 languages. M-GAAMA answers questions from a given passage in the same or a different language. It incorporates several existing multilingual models that can be used interchangeably in the demo such as M-BERT and XLM-R. The M-GAAMA demo also improves language accessibility by incorporating the IBM Watson machine translation widget to provide additional capabilities to the user to see an answer in their desired language. We also show how M-GAAMA can be used in downstream tasks by incorporating it into an END-TO-END-QA system using CFO (Chakravarti et al., 2019). We experiment with our system architecture on the Multi-Lingual Question Answering (MLQA) and the CORD-19 COVID (Wang et al., 2020;Tang et al., 2020) datasets to provide insights into the performance of the system.

GAAMA 2.0: An Integrated System That Answers Boolean and Extractive Questions

“…Our demo presents GAMMA 2.0 which adds the capability to answer both extractive and boolean questions in one integrated multilingual system. Other QA demos which do not support boolean questions include (Chakravarti et al 2019;Ferritto et al 2020;Yang et al 2019;Yang, Fang, and Lin 2017;Zhang et al 2021;Zhao and Lee 2020). Finally, our system backend achieves state-of-the-art results on the TYDI QA leaderboard.…”

“…Current machine reading comprehension (MRC) systems (Alberti, Lee, and Collins 2019;Chakravarti et al 2019;Ferritto et al 2020) typically feature a single model targeted at supplying short extractive answer spans, but boolean questions demand non-extractive YES/NO answers, as well as supporting evidence. We demonstrate here a system that, given a question, predicts the expected answer type, provides direct YES/NO answers with supporting evidence to boolean questions, and provides short answers to extractive questions.…”

GAAMA 2.0: An Integrated System that Answers Boolean and Extractive Question

“…Our demo is an extension of prior works GAAMA (Chakravarti et al, 2019) and M-GAAMA (Ferritto et al, 2020b). Both are QA demos with the former being English-only and the latter being crosslingual.…”

“…GAAMA is a single component that extracts a candidate answer span from the question/document pair. This component extends a traditional extractive question answering system (Ferritto et al, 2020b), and is implemented with a pointer network head on the 24 layer xlm-roberta-large (Conneau et al, 2020). We use multi-teacher knowledge distillation to distill the knowledge of both a TYDI QA model and a Natural Questions model into a single robust student model, using all training examples from the two datasets.…”

“…Current machine reading comprehension (MRC) systems Chakravarti et al, 2019;Ferritto et al, 2020b) typically feature a single model targeted at supplying short extractive answer spans, but boolean questions demand nonextractive yes/no answers, as well as supporting evidence. We demonstrate here a system that, given a question, predicts the expected answer type and provides direct YES/NO answers with supporting evidence to boolean questions, or provides short answers to extractive questions.…”