Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension

Abstract

Natural Questions is a new challenging machine reading comprehension benchmark with two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer). Despite the effectiveness of existing methods on this benchmark, they treat these two sub-tasks individually during training while ignoring their dependencies. To address this issue, we present a novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature, which are different levels of granularity: documents, paragraphs, sentences, and tokens. We utilize graph attention networks to obtain different levels of representations so that they can be learned simultaneously. The long and short answers can be extracted from paragraphlevel representation and token-level representation, respectively. In this way, we can model the dependencies between the two-grained answers to provide evidence for each other. We jointly train the two sub-tasks, and our experiments show that our approach significantly outperforms previous systems at both long and short answer criteria.

Does Structure Matter? Encoding Documents for Machine Reading Comprehension

“…However, such kind of linking information for creating triples is not necessarily prominent in documents other than Wikipedia. Some works segment the document content based on its semantic structures and rank them based on their relevance to the query (Yan et al, 2019;Lee et al, 2018;Wang et al, 2018;Zheng et al, 2020;.…”

Towards building a Robust Industry-scale Question Answering System

“…Architecturally, the latter is similar to us but we propose more technical novelty in terms of both improved attention and data augmentation. We note there is very recent academic work (Zheng et al, 2020)which we omit as GAAMA outperforms them on short answers and more importantly we compare against large scale industry SOTA for the scope of this paper. Since their work is more academic, their model enjoys being computationally more expensive for accuracy than GAAMA as they involve computing graph attentions that are typically more difficult to be run in parallel if we want to do whole graph propagation (Veličković et al, 2018).…”

Learn with Noisy Data via Unsupervised Loss Correction for Weakly Supervised Reading Comprehension

“…Machine Reading Comprehension. Machine reading comprehension (MRC) (Rajpurkar et al, 2016) has received increasing attention recently, which requires a model to extract an answer span to a question from reference documents (Yu et al, 2018;Devlin et al, 2019;Zheng et al, 2020;Yuan et al, 2020). Owing to the rise of pre-training models (Devlin et al, 2018), a machine is able to achieve highly competitive results on classic datasets (e.g.…”