Know What You Don't Know: Unanswerable Questions for SQuAD

Rajpurkar, Pranav; Jia, Robin; Liang, Percy

doi:10.48550/arxiv.1806.03822

Cited by 208 publications

(230 citation statements)

References 0 publications

Supporting

Mentioning

230

Contrasting

Order By: Relevance

“…Open-Domain Question Answering in NLP Opendomain question answering (Open-Domain QA) has been popular in the NLP community in recent years. The task is to answer a question with external knowledge bases without any given context paragraphs [43]. There are mainly two streams of approaches, namely knowledge graph-based question answering [10,31,37,51,58,64] and knowledge retrieval-based question answering [5,7,18,28,43,44,59,61].…”

Section: Related Workmentioning

confidence: 99%

“…The task is to answer a question with external knowledge bases without any given context paragraphs [43]. There are mainly two streams of approaches, namely knowledge graph-based question answering [10,31,37,51,58,64] and knowledge retrieval-based question answering [5,7,18,28,43,44,59,61]. For retrieval-based methods, both elastic-search such as BM25 [45] and semantic search such as Dense Passage Retrieval (DPR) [23] are utilized to retrieve most relevant knowledge snippets from knowledge bases.…”

Section: Related Workmentioning

confidence: 99%

“…For retrieval-based methods, both elastic-search such as BM25 [45] and semantic search such as Dense Passage Retrieval (DPR) [23] are utilized to retrieve most relevant knowledge snippets from knowledge bases. For question answering, most existing work adopt extractive methods to predict the span of an answer in knowledge snippets [5,7,28,43,44,59,61]. One most recent work proposes to use generative language models for knowledge-based QA, which achieves state-of-the-art performances [18].…”

Section: Related Workmentioning

confidence: 99%

“…First, although previous work on joint vision-language models formulates the task as an answer classification task [38,50,60], our preliminary studies show that language models seem to be less flexible in classifying text into such high-dimensional answer space (over 100k) given a relatively small dataset. Second, although most previous language QA models follow a span-based answer prediction paradigm [28,43,59,61], it is impractical in our opendomain setting since there is no ground-truth supporting fact in our task, let alone the ground-truth answer span for prediction. On the other hand, recent work shows that a generative encoder-decoder network can achieve state-of-theart performance on multiple open-domain QA datasets [41], and it avoids span prediction and directly generates a freeform answer.…”

Section: Generative Multi-passages Qamentioning

confidence: 99%

“…More specifically, the model needs to not only rank the retrieved passages but also predict an answer according to the ranked passages. Most existing work utilizes extractive methods to predict the answer span in the passage [5,7,28,43,44,59,61]. This is not applicable in the OK-VQA dataset because there is neither annotation of ground-truth passage nor answer span in any passage.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering

Gao¹,

Peng²,

Thattai³

et al. 2022

Preprint

View full text Add to dashboard Cite

Outside-knowledge visual question answering (OK-VQA) requires the agent to comprehend the image, make use of relevant knowledge from the entire web, and digest all the information to answer the question. Most previous works address the problem by first fusing the image and question in the multi-modal space, which is inflexible for further fusion with a vast amount of external knowledge. In this paper, we call for a paradigm shift for the OK-VQA task, which transforms the image into plain text, so that we can enable knowledge passage retrieval, and generative question-answering in the natural language space. This paradigm takes advantage of the sheer volume of gigantic knowledge bases and the richness of pretrained language models. A Transform-Retrieve-Generate framework (TRiG) framework is proposed 1 , which can be plug-and-played with alternative image-to-text models and textual knowledge bases. Experimental results show that our TRiG framework outperforms all state-of-the-art supervised methods by at least 11.1% absolute margin.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Generative Multi-passages Qamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering

Gao¹,

Peng²,

Thattai³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Question answering model based on machine reading comprehension with knowledge enhancement and answer verification

Yang

Sun

Kuang

2020

Concurrency and Computation

View full text Add to dashboard Cite

Summary Deep learning has led to important breakthroughs in natural language processing and obtained the state‐of‐the‐art results on machine reading comprehension. However, it is essential to consider the entity recognition and the detection of unanswerable questions for accuracy improvement. A novel question answering model is proposed with knowledge enhancement and answer verification to promote the performance of reading comprehension. With knowledge enhancement, the proposed model is able to recognize entities from the passage and detect word boundary precisely. To deal with unanswerable questions, the answerability of questions is evaluated based on the textual entailment. Empirical studies suggest that the proposed model has better ability of reading comprehension than others, with improvement on question answering tasks.

show abstract

How to Pre-train Your Model? Comparison of Different Pre-training Models for Biomedical Question Answering

Kamath

Grau

2020

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and finetuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training models based on reading comprehension and open domain question answering models and determine the performance when fine-tuned and tested over BIOASQ question answering dataset. We find open domain question answering model to be a better fit for this task rather than reading comprehension model.

show abstract

Know What You Don't Know: Unanswerable Questions for SQuAD

Cited by 208 publications

References 0 publications

A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering

A Thousand Words Are Worth More Than a Picture: Natural Language-Centric Outside-Knowledge Visual Question Answering

Question answering model based on machine reading comprehension with knowledge enhancement and answer verification

How to Pre-train Your Model? Comparison of Different Pre-training Models for Biomedical Question Answering

Contact Info

Product

Resources

About