Unsupervised Alignment-based Iterative Evidence Retrieval for Multi-hop Question Answering

Abstract

Evidence retrieval is a critical stage of question answering (QA), necessary not only to improve performance, but also to explain the decisions of the corresponding QA method. We introduce a simple, fast, and unsupervised iterative evidence retrieval method, which relies on three ideas: (a) an unsupervised alignment approach to soft-align questions and answers with justification sentences using only GloVe embeddings, (b) an iterative process that reformulates queries focusing on terms that are not covered by existing justifications, which (c) a stopping criterion that terminates retrieval when the terms in the given question and candidate answers are covered by the retrieved justifications. Despite its simplicity, our approach outperforms all the previous methods (including supervised methods) on the evidence selection task on two datasets: MultiRC and QASC. When these evidence sentences are fed into a RoBERTa answer classification component, we achieve state-of-the-art QA performance on these two datasets.

If You Want to Go Far Go Together: Unsupervised Joint Candidate Evidence Retrieval for Multi-hop Question Answering

“…(1) Impact of two-step evidence retrieval: Unsurprisingly, the two-step evidence retrieval process substantially impacts QA performance (e.g., row 1 vs. row 9 in table 2), which is consistent with the observations of previous works (Khot et al, 2019a;Yadav et al, 2020b). The top reranked WAIR chain leads to higher QA performance (+5.2% on QASC (row 12 vs. 13, table 2), and 2.3% F1 on MultiRC (row 14 vs. 15, table 4)).…”

“…Tables 2 and 4 list the main results for both question answering and evidence retrieval for the two datasets. Table 3 (Khot et al, 2019a) 73.2 --17 Two KF+SIR+2Step (Banerjee and Baral, 2020) 80.0 --18 Two AIR + RoBERTa (Yadav et al, 2020b) 81.0 --19 Two JointRR + RoBERTa 78.0 --Table 2: Question answering and evidence retrieval results on QASC. The second column indicates if the initial retrieval process is single step (e.g., a single iteration of BM25), or two steps (as in the WAIR approach).…”

Incorporating External Knowledge to Enhance Tabular Reasoning

“…To identify relevant rows, we employ a simplified version of the alignment algorithm used by Yadav et al (2019Yadav et al ( , 2020 for retrieval in reading comprehension.…”

Proceedings of the Fifteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-15)

“…Constructing long inference chains can be extremely challenging for existing models, which generally exhibit a large drop in performance when composing explanations and inference chains requiring more than 2 inference steps Khashabi et al, 156 2019;Yadav et al, 2020). To this end, this Shared Task on Multi-hop Inference for Explanation Regeneration (Jansen andUstalov, 2019, 2020) has focused on expanding the capacity of models to compose long inference chains, where participants are asked to develop systems capable of reconstructing detailed explanations for science exam questions drawn from the WorldTree explanation corpus , which range in compositional complexity from 1 to 16 facts (with the average explanation including 6 facts).…”