A Frame-based Sentence Representation for Machine Reading Comprehension

Abstract

Sentence representation (SR) is the most crucial and challenging task in Machine Reading Comprehension (MRC). MRC systems typically only utilize the information contained in the sentence itself, while human beings can leverage their semantic knowledge. To bridge the gap, we proposed a novel Frame-based Sentence Representation (FSR) method, which employs frame semantic knowledge to facilitate sentence modelling. Specifically, different from existing methods that only model lexical units (LUs), Frame Representation Models, which utilize both LUs in frame and Frame-to-Frame (F-to-F) relations, are designed to model frames and sentences with attention schema. Our proposed FSR method is able to integrate multiple-frame semantic information to get much better sentence representations. Our extensive experimental results show that it performs better than state-of-the-art technologies on machine reading comprehension task.

Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension

“…Thus, instead of simply mixing up them in sentence level, like above two methods, we design a novel Location-wise Fusion Method (LFM) to coherently integrate both syntax and frame semantic information at token level, obtaining a better sentence representation, shown in Figure 4. (Hermann et al, 2015) 46.3 41.9 Neural Reasoner (Peng et al, 2015) 47.6 45.6 Parallel-Hierarchical (Trischler et al, 2016) 74.58 71.00 Reading Strategies (Sun et al, 2018) 81.7 82.0 BERT+DCMN+ (Zhang et al, 2019) 85.0 86.5 XLNet+DCMN+ (Zhang et al, 2019) 86.2 86.6 FSR (Guo et al, 2020)…”

“…Input module takes in source context x and external feature text x sf , i.e., syntactic context x s and frame semantic context x f . In particular, syntactic context x s are produced by replacing words with their dependency labels, while frame semantic context x f are produced by replacing words with frames and frame elements (Guo et al, 2020). Then Bert (Devlin et al, 2018) is employed to encode the source context x into a vector g x .…”

“…Despite the success of those neural models, a number of studies have found there is a huge gap between MRC deep learning models and human beings (Wang and Jiang, 2019), as they might not really understand the natural language text (Mudrakarta et al, 2018). As such, some works employ semantic knowledge in neural network to facilitate sentence modelling (Guo et al, 2020;Zhang et al, 2018a;Zhang et al, 2018b). However, no work has been focused on integrating syntax and semantics into neural network for MRC.…”

Contextual and Semantic Fusion Network for Multiple-Choice Reading Comprehension

“…a buyer, French fries is a kind of food. It gives us a clue that semantic knowledge can help machine readers understand the meaning of a sentence [13].…”

“…To our knowledge, some studies have incorporated external semantic information to help their models better understand natural language text. Guo et al [13] employ semantic information by modeling lexical units based on the FrameNet [16] knowledge base whose semantic label form is completely different from the PropBank frame [17]. We adopt the PropBank-style semantic frame because it can cover every verb in a sentence.…”