Smarnet: Teaching Machines to Read and Comprehend Like Human

Abstract

Machine Comprehension (MC) is a challenging task in Natural Language Processing field, which aims to guide the machine to comprehend a passage and answer the given question. Many existing approaches on MC task are suffering the inefficiency in some bottlenecks, such as insufficient lexical understanding, complex question-passage interaction, incorrect answer extraction and so on. In this paper, we address these problems from the viewpoint of how humans deal with reading tests in a scientific way. Specifically, we first propose a novel lexical gating mechanism to dynamically combine the words and characters representations. We then guide the machines to read in an interactive way with attention mechanism and memory network. Finally we add a checking layer to refine the answer for insurance. The extensive experiments on two popular datasets SQuAD and TriviaQA show that our method exceeds considerable performance than most stateof-the-art solutions at the time of submission.

Multi-Mention Learning for Reading Comprehension with Neural Cascades

“…In the web domain, except for the verified F 1 scores, we see a similar trend. Surprisingly, we outperform approaches which use multi-layer recurrent pointer networks with specialized memories (Chen et al, 2017b;Hu et al, 2017) ) is significantly better than our final model. Although we found it could obtain high results, it was less consistent across different runs and gave lower scores on average (49.30) compared to our approach averaged over 4 runs (51.03).…”

“…Pointer networks with multi-hop reasoning, and syntactic and NER features, have been used recently in three architectures -Smarnet (Chen et al, 2017b), Reinforced Mnemonic Reader (Hu et al, 2017) and MEMEN (Pan et al, 2017) for both SQuAD and TriviaQA. Most of the above also use document truncation .…”

Neural Machine Reading Comprehension: Methods and Trends

“…Finally, the outputs of match-LSTM in two directions are concatenated together and are later fed to answer prediction module. In addition, R-Net [88], IA Reader [72] and Smarnet [8] also utilize RNNs to update the query-aware context representations to perform multi-hop interaction.…”

“…In the Smarnet model, Chen et al [8] not only use gate mechanism to control the question influence on the context, but also introduce another gate mechanism to refine query representations with the knowledge of context. The combination of these two gated-attention mechanisms implements the alternant reading between the context and question with mutual information.…”

A Fully Attention-Based Information Retriever

“…One possible explanation is that short passages give fewer options of simple questions, such as "when", "who", or "how many", and the annotators of the dataset had to resort to more elaborate alternatives. [42] Model EM (%) F1 (%) Reinforced Mnemonic Reader [3] 79.545 86.654 MEMEN [4] 78.234 85.344 FRC [32] 76.240 84.599 RaSoR + TR + LM [5] 77.583 84.163 Stochastic Answer Networks [6] 76.828 84.396 r-net [7] 76.461 84.265 FusionNet [8] 75.968 83.900 DCN+ [9] 75.087 83.081 Conductor-net [10] 74.405 82.742 BiDAF + Self Attention [11] 72.139 81.048 smartnet [12] 71.415 80.160 Ruminating Reader [13] 70.639 79.456 jNet [14] 70.607 79.821 ReasoNet [15] 70.555 79.364 Document Reader [16] 70.733 79.353 RaSoR [17] 70.849 78.741 FastQAExt [18] 70.849 78.857 Multi-Perspective Matching [19] 70.387 78.784 SEDT [20] 68.163 77.527 FABIR (Ours) 67.744 77.605 BiDAF [21] 67.974 77.323 Dynamic Coattention Networks [22] 66.233 77.896 Match-LSTM with Bi-Ans-Ptr [23] 64.744 73.743 Fine-Grained Gating [24] 62.446 73.327 OTF dict+spelling [25] 64.083 73.056 Dynamic Chunk Reader [26] 62.499 70.956…”