SQuAD: 100,000+ Questions for Machine Comprehension of Text

Rajpurkar, Pranav; Zhang, Jian; Lopyrev, Konstantin; Liang, Percy

doi:10.18653/v1/d16-1264

Cited by 4,334 publications

(4,300 citation statements)

References 22 publications

Supporting

Mentioning

4,277

Contrasting

Unclassified

Order By: Relevance

“…Single model EM / F1 EM / F1 LR Baseline (Rajpurkar et al, 2016) 40.0 / 51.0 40.4 / 51.0 Dynamic Chunk Reader (Yu et al, 2016) 62.5 / 71.2 62.5 / 71.0 Match-LSTM with Ans-Ptr (Wang and Jiang, 2016b) 64.1 / 73.9 64.7 / 73.7 Dynamic Coattention Networks (Xiong et al, 2016) 65.4 / 75.6 66.2 / 75.9 RaSoR (Lee et al, 2016) 66.4 / 74.9 -/ -BiDAF (Seo et al, 2016) 68.0 / 77.3 68.0 / 77.3 jNet (Zhang et al, 2017) -/ -68.7 / 77.4 Multi-Perspective Matching -/ -68.9 / 77.8 FastQA (Weissenborn et al, 2017) - (Rajpurkar et al, 2016) 80.3 / 90.5 77.0 / 86.8 Table 2: The performance of our gated self-matching networks (R-NET) and competing approaches 4 .…”

Section: Dev Set Test Setmentioning

confidence: 99%

See 1 more Smart Citation

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Wang

Yang

Wei

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

602

417

View full text Add to dashboard Cite

In this paper, we present the gated selfmatching networks for reading comprehension style question answering, which aims to answer questions from a given passage. We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation. Then we propose a self-matching attention mechanism to refine the representation by matching the passage against itself, which effectively encodes information from the whole passage. We finally employ the pointer networks to locate the positions of answers from the passages. We conduct extensive experiments on the SQuAD dataset. The single model achieves 71.3% on the evaluation metrics of exact match on the hidden test set, while the ensemble model further boosts the results to 75.9%. At the time of submission of the paper, our model holds the first place on the SQuAD leaderboard for both single and ensemble model.

show abstract

Section: Dev Set Test Setmentioning

confidence: 99%

“…Moreover, SQuAD requires different forms of logical reasoning to infer the answer (Rajpurkar et al, 2016). Rapid progress has been made since the release of the SQuAD dataset.…”

Section: Introductionmentioning

confidence: 99%

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Wang

Yang

Wei

et al. 2017

Proceedings of the 55th Annual Meeting of the Association For Computational Linguistics (Volume 1: Long Papers)

602

417

View full text Add to dashboard Cite

show abstract

“…There exist two big challenges: 1)Matching explicit information in the given context; 2)Incorporating implicit commonsense knowledge into human-like reasoning process. Previous machine comprehension tasks (Richardson et al, 2013;Rajpurkar et al, 2016) mainly focus on the first challenge, leading their solutions focusing on semantic matching between texts (Weston et al, 2014;Kumar et al, 2015;Narasimhan and Barzilay, 2015;Smith et al, 2015;Sukhbaatar et al, 2015;Hill et al, 2015;Wang et al, 2015Cui et al, 2016;Trischler et al, 2016a,b;Kadlec et al, 2016;Kobayashi et al, 2016;Wang and Jiang, 2016b), but ignore the second issues. One notable task is SNLI (Bowman et al, 2015), which considers entailment between two sentences.…”

Section: Related Workmentioning

confidence: 99%

Reasoning with Heterogeneous Knowledge for Commonsense Machine Comprehension

Lin¹,

Sun²,

Han³

2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Reasoning with commonsense knowledge is critical for natural language understanding.Traditional methods for commonsense machine comprehension mostly only focus on one specific kind of knowledge, neglecting the fact that commonsense reasoning requires simultaneously considering different kinds of commonsense knowledge. In this paper, we propose a multi-knowledge reasoning method, which can exploit heterogeneous knowledge for commonsense machine comprehension. Specifically, we first mine different kinds of knowledge (including event narrative knowledge, entity semantic knowledge and sentiment coherent knowledge) and encode them as inference rules with costs. Then we propose a multiknowledge reasoning model, which selects inference rules for a specific reasoning context using attention mechanism, and reasons by summarizing all valid inference rules. Experiments on RocStories show that our method outperforms traditional models significantly.

show abstract

“…Following the recent progress on end-to-end supervised question answering (Hermann et al, 2015;Rajpurkar et al, 2016), we consider the general problem of predicting an answer A given a query-document pair (Q, D). We do not make the assumption that the answer should be present verbatim in the document.…”

Section: Problem Descriptionmentioning

confidence: 99%

“…The dataset contains 18.58M instances divided into training, validation, and test with an 85/10/5 split. The answer is present verbatim in the document only 47.1% of the time, severely limiting models that label document spans, such as those developed for the popular SQUAD dataset (Rajpurkar et al, 2016).…”

Section: Supervised Versionmentioning

confidence: 99%

Accurate Supervised and Semi-Supervised Machine Reading for Long Documents

Hewlett¹,

Jones²,

Lacoste³

et al. 2017

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

We introduce a hierarchical architecture for machine reading capable of extracting precise information from long documents. The model divides the document into small, overlapping windows and encodes all windows in parallel with an RNN. It then attends over these window encodings, reducing them to a single encoding, which is decoded into an answer using a sequence decoder. This hierarchical approach allows the model to scale to longer documents without increasing the number of sequential steps. In a supervised setting, our model achieves state of the art accuracy of 76.8 on the WikiReading dataset. We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows. We evaluate models that can reuse autoencoder states and outputs without finetuning their weights, allowing for more efficient training and inference.

show abstract

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Cited by 4,334 publications

References 22 publications

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Gated Self-Matching Networks for Reading Comprehension and Question Answering

Reasoning with Heterogeneous Knowledge for Commonsense Machine Comprehension

Accurate Supervised and Semi-Supervised Machine Reading for Long Documents

Contact Info

Product

Resources

About