A question-entailment approach to question answering

Abstract

BackgroundOne of the challenges in large-scale information retrieval (IR) is developing fine-grained and domain-specific methods to answer natural language questions. Despite the availability of numerous sources and datasets for answer retrieval, Question Answering (QA) remains a challenging problem due to the difficulty of the question understanding and answer extraction tasks. One of the promising tracks investigated in QA is mapping new questions to formerly answered questions that are “similar”.ResultsWe propose a novel QA approach based on Recognizing Question Entailment (RQE) and we describe the QA system and resources that we built and evaluated on real medical questions. First, we compare logistic regression and deep learning methods for RQE using different kinds of datasets including textual inference, question similarity, and entailment in both the open and clinical domains. Second, we combine IR models with the best RQE method to select entailed questions and rank the retrieved answers. To study the end-to-end QA approach, we built the MedQuAD collection of 47,457 question-answer pairs from trusted medical sources which we introduce and share in the scope of this paper. Following the evaluation process used in TREC 2017 LiveQA, we find that our approach exceeds the best results of the medical task with a 29.8% increase over the best official score.ConclusionsThe evaluation results support the relevance of question entailment for QA and highlight the effectiveness of combining IR and RQE for future QA efforts. Our findings also show that relying on a restricted set of reliable answer sources can bring a substantial improvement in medical QA.

A Systematic Review of Automatic Question Generation for Educational Purposes

“…Ensuring that questions do not overlap or provide clues for other questions also needs to be taken into account. The AQG field could adopt ideas from the question answering field in which question entailment has been investigated (for example, see the work of Abacha and Demner-Fushman (2016)). Finally, ordering questions in a way that increases motivation and maximises the accuracy of scores is another interesting area.…”

Lessons from Natural Language Inference in the Clinical Domain

“…Previous research relevant to the present topic, is the work on RTE in the biomedical domain: automatic construction of textual entailment datasets (Abacha et al, 2015;Abacha and Demner-Fushman, 2016), use of active learning on limited RTE data (Shivade et al, 2015(Shivade et al, , 2016, and enhancement of search results (Adler et al, 2012) using TE models. These efforts were limited due to the above-mentioned constraints.…”

HEAD-QA: A Healthcare Dataset for Complex Reasoning

“…Which of the following tumors is more likely to be present? However, this kind of dataset is scarce for complex domains like medicine: while challenges have been proposed in such domains, like textual entailment (Abacha et al, 2015;Abacha and Dina, 2016) or answering questions about specific documents and snippets (Nentidis et al, 2018), we know of no resources that require general reasoning on complex domains. The novelty of this work falls in this direction, presenting a multi-choice QA task that combines the need of knowledge and reasoning with complex domains, and which takes humans years of training to answer correctly.…”