Four-way Bidirectional Attention for Multiple-choice Reading Comprehension

Hu, Lei; Zou, Dongsheng; Guo, Xiwang; Qi, Liang; Tang, Ying; Song, Haohao; Yuan, Jieying

doi:10.1109/smc52423.2021.9658632

Cited by 7 publications

(7 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multiway attention (Wang et al, 2018a;Zhu et al, 2018) has been shown to be effective in capturing the interactions between each pair of input paragraph, question and candidate answers, leading to better context interpretation, while BERT fine-tuning (Devlin et al, 2018) also shows its prominent ability in commonsense inference. To further enhance the context understanding ability of BERT fine-tuning, we perform multiway bidirectional attention over the BERT encoding output.…”

Section: Bert With Multiway Attentionmentioning

confidence: 99%

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning

Huang¹,

Bras²,

Bhagavatula³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

203

179

View full text Add to dashboard Cite

Understanding narratives requires reading between the lines, which in turn, requires interpreting the likely causes and effects of events, even when they are not mentioned explicitly. In this paper, we introduce COSMOS QA, a large-scale dataset of 35, 600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions. In stark contrast to most existing reading comprehension datasets where the questions focus on factual and literal understanding of the context paragraph, our dataset focuses on reading between the lines over a diverse collection of people's everyday narratives, asking such questions as "what might be the possible reason of ...?", or "what would have happened if ..." that require reasoning beyond the exact text spans in the context. To establish baseline performances on COSMOS QA, we experiment with several state-of-the-art neural architectures for reading comprehension, and also propose a new architecture that improves over the competitive baselines. Experimental results demonstrate a significant gap between machine (68.4%) and human performance (94%), pointing to avenues for future research on commonsense machine comprehension. Dataset, code and leaderboard is publicly available at https://wilburone.

show abstract

Section: Bert With Multiway Attentionmentioning

confidence: 99%

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning

Huang¹,

Bras²,

Bhagavatula³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

203

179

View full text Add to dashboard Cite

show abstract

“…Yet in a task like Social IQA, there is a distinct separation of context, question, and answer, and we would like to emphasize the differences between these segments. Previous work has dealt with this property with different architectures, usually using complex cross-segment (or named multiway) attention blocks [2,21,22,23].…”

Section: Cross-segment Attentionmentioning

confidence: 99%

Go Beyond Plain Fine-Tuning: Improving Pretrained Models for Social Commonsense

Chang

Liu

Gopalakrishnan

et al. 2021

2021 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Pretrained language models have demonstrated outstanding performance in many NLP tasks recently. However, their social intelligence, which requires commonsense reasoning about the current situation and mental states of others, is still developing. Towards improving language models' social intelligence, in this study we focus on the Social IQA dataset, a task requiring social and emotional commonsense reasoning. Building on top of the pretrained RoBERTa and GPT2 models, we propose several architecture variations and extensions, as well as leveraging external commonsense corpora, to optimize the model for Social IQA. Our proposed system achieves competitive results as those top-ranking models on the leaderboard. This work demonstrates the strengths of pretrained language models, and provides viable ways to improve their performance for a particular task.

show abstract

“…and model the semantic relationships among paragraph, question and candidate options from multiple aspects of matching. Zhu et al (2018a) propose a hierarchical attention flow model, which leverages candidate options to capture the interactions among paragraph, question and candidate options. merge various attentions to fully extract the mutual information among the paragraph, question and options and form the enriched representations.…”

Section: Multi-choice Reading Comprehensionmentioning

confidence: 99%

Commonsense Evidence Generation and Injection in Reading Comprehension

Liu¹,

Yang²,

You³

et al. 2020

Preprint

View full text Add to dashboard Cite

Human tackle reading comprehension not only based on the given context itself but often rely on the commonsense beyond. To empower the machine with commonsense reasoning, in this paper, we propose a Commonsense Evidence Generation and Injection framework in reading comprehension, named CEGI. The framework injects two kinds of auxiliary commonsense evidence into comprehensive reading to equip the machine with the ability of rational thinking. Specifically, we build two evidence generators: the first generator aims to generate textual evidence via a language model; the other generator aims to extract factual evidence (automatically aligned text-triples) from a commonsense knowledge graph after graph completion. Those evidences incorporate contextual commonsense and serve as the additional inputs to the model. Thereafter, we propose a deep contextual encoder to extract semantic relationships among the paragraph, question, option, and evidence. Finally, we employ a capsule network to extract different linguistic units (word and phrase) from the relations, and dynamically predict the optimal option based on the extracted units. Experiments on the CosmosQA dataset demonstrate that the proposed CEGI model outperforms the current state-of-the-art approaches and achieves the accuracy (83.6%) on the leaderboard.

show abstract

Four-way Bidirectional Attention for Multiple-choice Reading Comprehension

Cited by 7 publications

References 11 publications

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning

Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning

Go Beyond Plain Fine-Tuning: Improving Pretrained Models for Social Commonsense

Commonsense Evidence Generation and Injection in Reading Comprehension

Contact Info

Product

Resources

About