Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretable

Abstract

Multi-hop QA requires the machine to answer complex questions through finding multiple clues and reasoning, and provide explanatory evidence to demonstrate the machine's reasoning process. We propose Relation Extractor-Reader and Comparator (RERC), a three-stage framework based on complex question decomposition. The Relation Extractor decomposes the complex question, and then the Reader answers the sub-questions in turn, and finally the Comparator performs numerical comparison and summarizes all to get the final answer, where the entire process itself constitutes a complete reasoning evidence path. In the 2WikiMultiHopQA dataset, our RERC model has achieved the state-of-the-art performance, with a winning joint F1 score of 53.58 on the leaderboard. All indicators of our RERC are close to human performance, with only 1.95 behind the human level in F1 score of support fact. At the same time, the evidence path provided by our RERC framework has excellent readability and faithfulness.

NA‐Reviewer: Reviewing the context to improve the error accumulation issue for multi‐hop QA

“…Following the setting in [1], we use Adam optimizer and set the $$batch \: size$$ to 32 and the learning rate to $$2\times 10^{-5}$$, which decreases linearly during the training process. For NA‐Discriminator module and Reviewer module, we use the sequence length $$l=512$$ and hidden size $$d=768$$.…”

“…The RERC model proposed in our previous work [1] is a three‐stage multi‐hop QA model based on question decomposition, which consists of three parts: Relation Extractor, Reader and Comparator.…”

“…Some end‐to‐end methods [6, 7] have been proposed and excellent performance has been achieved on many multi‐hop QA tasks, but they also exposed many problems. First of all, the internal reasoning mechanism of those end‐to‐end methods is not clear, and some explanatory works are not faithful enough [1]. Second, [5] shows that the joint training model may hurt each stage on a specific task.…”

Ask to Understand: Question Generation for Multi-hop Question Answering

“…Then the graph neural network that builds graphs based on entities was introduced to multi-hop QA tasks and achieved astonishing performance (De Cao et al, 2019;Tu et al, 2019;Ding et al, 2019). While, some researchers paid much attention to the interpretability of the coreference reasoning chains (Fu et al, 2021;Nishida et al, 2019;Min et al, 2019;Jiang and Bansal, 2019b). By providing decomposed single-hop sub-questions, the QD-based method makes the model decisions explainable.…”

“…Inspired by that human solves such questions by following a transparent and explainable logical route, another popular stream of Question Decomposition-based (QD) approaches became favored in recent years (Fu et al, 2021;Nishida et al, 2019;Min et al, 2019;Jiang and Bansal, 2019b). The method mimics human reasoning to decompose complex questions into simpler, single-hop sub-questions; thus, the interpretability is greatly improved by exposing intermediate evidence generated by each sub-question.…”