Question Answering by Reasoning Across Documents with Graph Convolutional Networks

Cao, Nicola De; Aziz, Wilker; Titov, Ivan

doi:10.48550/arxiv.1808.09920

Cited by 23 publications

(45 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As multi-hop relations in KBs could be mentioned together in a single text piece, these datasets are not designed for an open-domain setting which necessitates multi-hop retrieval. Existing methods on these datasets either retrieve passages from a small passage pool pruned based on the the specific dataset Dhingra et al, 2020), or focus on a non-retrieval setting where a compact documents set is already given (De Cao et al, 2018;Tu et al, 2019;Beltagy et al, 2020). Compared to these research, our work aims at building an efficient multi-hop retrieval model that easily scales to large real-world corpora that include millions of open-domain documents.…”

Section: Related Workmentioning

confidence: 99%

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Xiong

Iyer

et al. 2020

Preprint

View full text Add to dashboard Cite

We propose a simple and efficient multi-hop dense retrieval approach for answering complex open-domain questions, which achieves state-of-the-art performance on two multi-hop datasets, HotpotQA and multi-evidence FEVER. Contrary to previous work, our method does not require access to any corpus-specific information, such as inter-document hyperlinks or human-annotated entity markers, and can be applied to any unstructured text corpus. Our system also yields a much better efficiency-accuracy trade-off, matching the best published accuracy on HotpotQA while being 10 times faster at inference time.

show abstract

Section: Related Workmentioning

confidence: 99%

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Xiong

Iyer

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Graph Neural Networks Graph neural networks (GNNs) capture the dependencies and relations between nodes connected with edges, which propagate features across nodes layer by layer (Scarselli et al, 2008;Kipf & Welling, 2016;Hamilton et al, 2017). GNNs have demonstrated effectiveness in a wide variety of tasks in natural language processing such as text classification (Yao et al, 2019;, machine translation (Bastings et al, 2017), question answering (Song et al, 2018;De Cao et al, 2018), recommendation (Wu et al, 2019) and information extraction (Li et al, 2020a). For example, proposed Star Transformer, a Transformer backbone but replaces the fully-connected structure in self-attention with a star-like topology, in which every two non-adjacent nodes are connected through a shared relay node.…”

Section: Related Workmentioning

confidence: 99%

GNN-LM: Language Modeling based on Global Contexts via GNN

Meng¹,

Zong²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Inspired by the notion that "to copy is easier than to memorize", in this work, we introduce GNN-LM, which extends vanilla neural language model (LM) by allowing to reference similar contexts in the entire training corpus. We build a directed heterogeneous graph between an input context and its semantically related neighbors selected from the training corpus, where nodes are tokens in the input context and retrieved neighbor contexts, and edges represent connections between nodes. Graph neural networks (GNNs) are constructed upon the graph to aggregate information from similar contexts to decode the token. This learning paradigm provides direct access to the reference contexts and helps improve a model's generalization ability. We conduct comprehensive experiments to validate the effectiveness of the GNN-LM: GNN-LM achieves a new state-of-the-art perplexity of 14.8 on WikiText-103 (a 4.5 point improvement over its counterpart of the vanilla LM model), and shows substantial improvement on One Billion Word and Enwiki8 datasets against strong baselines. In-depth ablation studies are performed to understand the mechanics of GNN-LM. 1

show abstract

“…A sequential approach is followed by Memory Network-based models to iteratively store the information gathered from passages in a memory cell [22,34,38]. Works by [4,11,36] use graph convolutional network [21] to do multi-hop reasoning.…”

Section: Multi-step Datasets and Reasoningmentioning

confidence: 99%

Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering

Liu,

Chowdhury,

Zhang

et al. 2020

Preprint

View full text Add to dashboard Cite

Healthcare question answering assistance aims to provide customer healthcare information, which widely appears in both Web and mobile Internet. The questions usually require the assistance to have proficient healthcare background knowledge as well as the reasoning ability on the knowledge. Recently a challenge involving complex healthcare reasoning, HeadQA dataset, has been proposed, which contains multiple choice questions authorized for the public healthcare specialization exam. Unlike most other QA tasks that focus on linguistic understanding, HeadQA requires deeper reasoning involving not only knowledge extraction, but also complex reasoning with healthcare knowledge. These questions are the most challenging for current QA systems, and the current performance of the state-of-the-art method is slightly better than a random guess. In order to solve this challenging task, we present a Multi-step reasoning with Knowledge extraction framework (MurKe). The proposed framework first extracts the healthcare knowledge as supporting documents from the large corpus. In order to find the reasoning chain and choose the correct answer, MurKe iterates between selecting the supporting documents, reformulating the query representation using the supporting documents and getting entailment score for each choice using the entailment model. The reformulation module leverages selected documents for missing evidence, which maintains interpretability. Moreover, we are striving to make full use of off-the-shelf pretrained models. With less trainable weight, the pretrained model can easily adapt to healthcare tasks with limited training samples. From the experimental results and ablation study, our system is able to outperform several strong baselines on the HeadQA dataset.

show abstract

Question Answering by Reasoning Across Documents with Graph Convolutional Networks

Cited by 23 publications

References 0 publications

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval

GNN-LM: Language Modeling based on Global Contexts via GNN

Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering

Contact Info

Product

Resources

About