A Vietnamese Dataset for Evaluating Machine Reading Comprehension

Abstract

Over 97 million people speak Vietnamese as their native language in the world. However, there are few research studies on machine reading comprehension (MRC) for Vietnamese, the task of understanding a text and answering questions related to it. Due to the lack of benchmark datasets for Vietnamese, we present the Vietnamese Question Answering Dataset (UIT-ViQuAD), a new dataset for the low-resource language as Vietnamese to evaluate MRC models. This dataset comprises over 23,000 human-generated question-answer pairs based on 5,109 passages of 174 Vietnamese articles from Wikipedia. In particular, we propose a new process of dataset creation for Vietnamese MRC. Our in-depth analyses illustrate that our dataset requires abilities beyond simple reasoning like word matching and demands single-sentence and multiple-sentence inferences. Besides, we conduct experiments on state-of-the-art MRC methods for English and Chinese as the first experimental models on UIT-ViQuAD. We also estimate human performance on the dataset and compare it to the experimental results of powerful machine learning models. As a result, the substantial differences between human performance and the best model performance on the dataset indicate that improvements can be made on UIT-ViQuAD in future research. Our dataset is freely available on our website 1 to encourage the research community to overcome challenges in Vietnamese MRC.

Conversational Machine Reading Comprehension for Vietnamese Healthcare Texts

“…For the machine reading comprehension model, the Document Reader (DrQA) introduced by Chen et al [1] is a powerful model on various of machine reading comprehension corpora such as: SQuAD [11], TextWorldsQA [8], and UIT-ViSQuAD [10]. The DrQA model consists of two modules: Document Retriever and Document Reader.…”

“…Many of MRC corpora are constructed on specific domains and open domains in English such as SQuAD [11] (extractive MRC) on Wikipedia articles, RACE [4] (multiple choices MRC) on High school students English Exams domain, and NarrativeQA [7] (abstractive MRC) on books and stories domain. For the Vietnamese language, the UIT-ViQuAD [10] (Wikipedia domain) and ViNewQA [15] (Health news domain) are two extractive MRC corpora for machine reading comprehension. Besides, the ViMMRC [9] is the multiple-choice reading comprehension corpus on the Vietnamese students' textbook for primary schools domain.…”

QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

“…Three more languages have their versions of SQuAD [210]: French [66,126], Vietnamese [187], and Korean [150],…”

A term-based and citation network-based search system for COVID-19

“…The output of Pyserini is then reranked by a T5 language model, 10 which is fine-tuned on MS MARCO, a large machine reading comprehension dataset. 18 Similarly, SLEDGE 19 uses a similar approach, but using SciBERT 13 to rerank documents.…”