Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehension

Abstract

This paper describes our system for SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge (Ostermann et al., 2018b).We use Threeway Attentive Networks (TriAN) to model interactions between the passage, question and answers. To incorporate commonsense knowledge, we augment the input with relation embedding from the graph of general knowledge ConceptNet (Speer et al., 2017). As a result, our system achieves state-of-the-art performance with 83.95% accuracy on the official test data. Code is publicly available at https://github.com/ intfloat/commonsense-rc.

Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning

“…We compare our single model with other existing systems (single model). We also adopt relation-aware tasks on the co-attention layer of the TriAN model [12]. From the result, we can observe that: (1) Our method achieves better performance on both datasets compared with previous methods and Bert(base) model.…”

“…From the result, we can observe that: (1) Our method achieves better performance on both datasets compared with previous methods and Bert(base) model. (2) By adopt relation-aware tasks on the attention layer of the TriAN model [12] on SemEval, the model performance can also be improved. The results show that the relation-aware tasks can help to better align sentences due to knowledge gap.…”

Improving Machine Reading Comprehension with General Reading Strategies

“…We follow the philosophy of transferring the knowledge from a high-performing model pretrained on a large-scale supervised data of a source task to a target task, in which only a small amount of training data is available (Chung et al, 2018). RACE has been used to pre-train a model for other MRC tasks as it contains the largest number of general domain non-extractive questions (Table 1) (Ostermann et al, 2018;Wang et al, 2018a). In our experiment, we also treat RACE as the source task and regard six representative non-extractive multiple-choice MRC datasets from multiple domains as the target tasks.…”

Improving Subject-Area Question Answering with External Knowledge

“…In our experiment, we use RACE (Lai et al, 2017) -the largest existing multiple-choice machine reading comprehension dataset collected from real and practical language exams -in the pre-finetuning stage. Questions in RACE focus on evaluating linguistic knowledge acquisition of participants and are commonly used in previous methods (Wang et al, 2018a;Sun et al, 2019). We evaluate the performance of our methods on three multiple-choice science QA datasets: ARC-Easy, ARC-Challenge, and OpenBookQA.…”