PersianQuAD: The Native Question Answering Dataset for the Persian Language

Abstract

Developing Question Answering systems (QA) is one of the main goals in Artificial Intelligence. With the advent of Deep Learning (DL) techniques, QA systems have witnessed significant advances. Although DL performs very well on QA, it requires a considerable amount of annotated data for training. Many annotated datasets have been built for the QA task; most of them are exclusively in English. In order to address the need for a high-quality QA dataset in the Persian language, we present PersianQuAD, the native QA dataset for the Persian language. We create PersianQuAD in four steps: (1) Wikipedia article selection, ( 2) question-answer collection, (3) three-candidates test set preparation, and (4) Data Quality Monitoring. PersianQuAD consists of approximately 20,000 questions and answers made by native annotators on a set of Persian Wikipedia articles. The answer to each question is a segment of the corresponding article text. To better understand PersianQuAD and ensure its representativeness, we analyze PersianQuAD and show it contains questions of varying types and difficulties. We also present three versions of a deep learning-based QA system trained with PersianQuAD. Our best system achieves an F1 score of 82.97% which is comparable to that of QA systems on English SQuAD, made by the Stanford University. This shows that PersianQuAD performs well for training deep-learning-based QA systems. Human performance on PersianQuAD is significantly better (96.49%), demonstrating that PersianQuAD is challenging enough and there is still plenty of room for future improvement. PersianQuAD is freely available and can be downloaded from here. All the QA systems implemented in this paper are also available here.

Slovak Dataset for Multilingual Question Answering

“…Manual creation of questions and marking of answers to relevant paragraphs was usually done by crowdsourcing by trained workers. To create answers to the questions, either own annotation tools were used, such as AddQA [45], PI-AFanno [46], SAJAD [47], or already existing web crowdsourcing platforms such as Amazon Mechanical Turk [28], Toloka AI [40], or Prolific [48].…”

A Persian Benchmark for Joint Intent Detection and Slot Filling

“…There are Persian datasets for NLP tasks like questionanswering [12], [13], [14], language modeling [19], or sentiment analysis [20]. However, there is no Persian benchmark dataset for the NLU task.…”

PerAnSel: A Novel Deep Neural Network-Based System for Persian Question Answering

“…There are some machine reading comprehension datasets for Persian [ 66 , 67 ]. We build PASD by using the PersianQuAD dataset [ 67 ].…”

“…There are some machine reading comprehension datasets for Persian [ 66 , 67 ]. We build PASD by using the PersianQuAD dataset [ 67 ]. PersianQuAD is the first large-scale native machine reading comprehension dataset for question answering for the Persian language.…”