SQuAD: 100,000+ Questions for Machine Comprehension of Text

Rajpurkar, Pranav; Zhang, Jian; Lopyrev, Konstantin; Liang, Percy

doi:10.48550/arxiv.1606.05250

Cited by 664 publications

(817 citation statements)

References 21 publications

Supporting

Mentioning

814

Contrasting

Unclassified

Order By: Relevance

“…In principle, as a dual task of QA, any QA datasets can be used for QG [50]. SQuAD [58], MS-MARCO [4] and newsQA [73] are three famous datasets used for answer-extraction QG, collected from Wikipedia, Bing search logs, and CNN news respectively. Unlike the previous three datasets, Nar-rativeQA [35] does not restrict the answers to be the span of texts in the articles, therefore, it can be used as an answer-abstraction QG dataset.…”

Section: Related Work 21 Question Generationmentioning

confidence: 99%

“…Chan et al build a recurrent BERT to output one question word at a recurrent step [11,12], but it is time-consuming. The generative pretrained models such as UNILM [18], T5 [57], PEGASUS [82], and UNILMV2 [5] report the model's QG scores finetuned on SQuAD [58] dataset, but they do not explore the idea of building a unified QG.…”

Section: Related Work 21 Question Generationmentioning

confidence: 99%

“…• Answer-extraction QG: SQuADv1.1 [58] • Answer-abstraction QG: NarrativeQA [35] • Multi-choice QG: RACE [38], McTest [64], OpenbookQA [49], ARC-easy, ARC-hard [15] • Boolean QG: BoolQA [14] We assume datasets arrive in the following order: "McTest→SQuAD →RACE→NarrativeQA→Arc-easy→Arc-hard→OpenbookQA →BoolQA", which corresponds to the exact release dates of these datasets in the real world. Details on dataset characteristics, statistics, and splitting strategies are in Appendix A.1.…”

Section: Experiments 51 Datasetsmentioning

confidence: 99%

“…SQuAD is one of the most commonly used answer-extraction QG datasets. There are two versions, and we use SQuADv1.1 [58] in our experiment, which contains 536 Wikipedia articles with more than 100k questions. Following [81], we split the dataset into training, dev, and testing sets, each set containing 87, 599, 5, 286, and 5, 285 elements.…”

Section: A Implementation Details A1 Dataset Detailsmentioning

confidence: 99%

See 3 more Smart Citations

Unified Question Generation with Continual Lifelong Learning

Yuan,

Yin,

et al. 2022

Preprint

View full text Add to dashboard Cite

Question Generation (QG), as a challenging Natural Language Processing task, aims at generating questions based on given answers and context. Existing QG methods mainly focus on building or training models for specific QG datasets. These works are subject to two major limitations: (1) They are dedicated to specific QG formats (e.g., answer-extraction or multi-choice QG), therefore, if we want to address a new format of QG, a re-design of the QG model is required. (2) Optimal performance is only achieved on the dataset they were just trained on. As a result, we have to train and keep various QG models for different QG datasets, which is resource-intensive and ungeneralizable.To solve the problems, we propose a model named Unified-QG based on lifelong learning techniques, which can continually learn QG tasks across different datasets and formats. Specifically, we first build a format-convert encoding to transform different kinds of QG formats into a unified representation. Then, a method named STRIDER (SimilariT y RegularI zed Difficult Example Replay) is built to alleviate catastrophic forgetting in continual QG learning. Extensive experiments were conducted on 8 QG datasets across 4 QG formats (answer-extraction, answer-abstraction, multi-choice, and boolean QG) to demonstrate the effectiveness of our approach. Experimental results demonstrate that our Unified-QG can effectively and continually adapt to QG tasks when datasets and formats vary. In addition, we verify the ability of a single trained Unified-QG model in improving 8 Question Answering (QA) systems' performance through generating synthetic QA data.

show abstract

Section: Related Work 21 Question Generationmentioning

confidence: 99%

Section: Related Work 21 Question Generationmentioning

confidence: 99%

Section: Experiments 51 Datasetsmentioning

confidence: 99%

Section: A Implementation Details A1 Dataset Detailsmentioning

confidence: 99%

See 2 more Smart Citations

Unified Question Generation with Continual Lifelong Learning

Yuan,

Yin,

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We conduct experiments on the General Language Understanding Evaluation (GLUE) benchmark (Wang et al, 2018). We compare our method with the baseline methods on two single-sentence classification tasks (CoLA (Warstadt et al, 2018), SST-2 (Socher et al, 2013)), two similarity and paraphrase tasks (MRPC (Dolan & Brockett, 2005), QQP (Chen et al, 2018)), and three inference tasks (MNLI (Williams et al, 2018), QNLI (Rajpurkar et al, 2016), RTE (Dagan et al, 2005;Haim et al, 2006;Giampiccolo et al, 2007;Bentivogli et al, 2009)) 1 . We report accuracy for MNLI, QNLI, QQP, SST-2, RTE, report f1 for MRPC, and report Matthew's correlation for CoLA.…”

Section: Setupmentioning

confidence: 99%

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Xu¹,

Mukherjee²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Knowledge distillation (KD) methods compress large models into smaller students with manuallydesigned student architectures given pre-specified computational cost. This requires several trials to find a viable student, and further repeating the process for each student or computational budget change. We use Neural Architecture Search (NAS) to automatically distill several compressed students with variable cost from a large model. Current works train a single SuperLM consisting of millions of subnetworks with weight-sharing, resulting in interference between subnetworks of different sizes. Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e.g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training. Fully task-agnostic training and search allow students to be reused for fine-tuning on any downstream task. Experiments on GLUE benchmark against state-of-the-art KD and NAS methods demonstrate AutoDistil to outperform leading compression techniques with upto 2.7x reduction in computational cost and negligible loss in task performance.

show abstract

Question answering model based on machine reading comprehension with knowledge enhancement and answer verification

Yang

Sun

Kuang

2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Deep learning has led to important breakthroughs in natural language processing and obtained the state‐of‐the‐art results on machine reading comprehension. However, it is essential to consider the entity recognition and the detection of unanswerable questions for accuracy improvement. A novel question answering model is proposed with knowledge enhancement and answer verification to promote the performance of reading comprehension. With knowledge enhancement, the proposed model is able to recognize entities from the passage and detect word boundary precisely. To deal with unanswerable questions, the answerability of questions is evaluated based on the textual entailment. Empirical studies suggest that the proposed model has better ability of reading comprehension than others, with improvement on question answering tasks.

show abstract

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Cited by 664 publications

References 21 publications

Unified Question Generation with Continual Lifelong Learning

Unified Question Generation with Continual Lifelong Learning

AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models

Question answering model based on machine reading comprehension with knowledge enhancement and answer verification

Contact Info

Product

Resources

About