DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension

Zhang, Shuailiang; Zhang, Hai; Wu, Yuwei; Zhang, Zhuosheng; Zhou, Xi; Zhou, Xiang

doi:10.1609/aaai.v34i05.6502

Cited by 105 publications

(55 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RACE For RACE, we compare our model with the following latest baselines: Dual Co-Matching Network (DCMN) (Zhang et al 2020a), Option Comparison Network (OCN) (Ran et al 2019), Reading Strategies Model (RSM) (Sun et al 2018), and Generative Pre-Training (GPT) (Radford et al 2018). Table 2 shows the result 7 .…”

Section: Resultsmentioning

confidence: 99%

“…Recently, much progress has been made in general-purpose language modeling that can be used across a wide range of tasks (Radford et al 2018;Devlin et al 2018;Zhang et al 2020b;Zhou, Zhang, and Zhao 2019;. Understanding the meaning of a sentence is a prerequisite to solve many natural language understanding (NLU) problems, such as machine reading comprehension (MRC) based question answering (Rajpurkar, Jia, and Liang 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SG-Net: Syntax-Guided Machine Reading Comprehension

Zhang

Zhou

et al. 2020

AAAI

Self Cite

138

View full text Add to dashboard Cite

For machine reading comprehension, the capacity of effectively modeling the linguistic knowledge from the detail-riddled and lengthy passages and getting ride of the noises is essential to improve its performance. Traditional attentive models attend to all words without explicit constraint, which results in inaccurate concentration on some dispensable words. In this work, we propose using syntax to guide the text modeling by incorporating explicit syntactic constraints into attention mechanism for better linguistically motivated word representations. In detail, for self-attention network (SAN) sponsored Transformer-based encoder, we introduce syntactic dependency of interest (SDOI) design into the SAN to form an SDOI-SAN with syntax-guided self-attention. Syntax-guided network (SG-Net) is then composed of this extra SDOI-SAN and the SAN from the original Transformer encoder through a dual contextual architecture for better linguistics inspired representation. To verify its effectiveness, the proposed SG-Net is applied to typical pre-trained language model BERT which is right based on a Transformer encoder. Extensive experiments on popular benchmarks including SQuAD 2.0 and RACE show that the proposed SG-Net design helps achieve substantial performance improvement over strong baselines.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

SG-Net: Syntax-Guided Machine Reading Comprehension

Zhang

Zhou

et al. 2020

AAAI

Self Cite

138

View full text Add to dashboard Cite

show abstract

“…Recent MCRC systems based on the pre-trained language model also focus on modeling the relationship among passage, question, and option. Zhang et al [30] model the passage-question, passage-option, and question-option pairwise relationship simultaneously and bidirectionally for each triplet. Imitating human beings, Ran et al [31] compare options with question information to identify the options correlations, and then reread the passage with the option correlation features.…”

Section: B Methods For Multiple-choice Machine Reading Comprehensionmentioning

confidence: 99%

Contextual and Semantic Fusion Network for Multiple-Choice Reading Comprehension

Duan

Huang

2021

IEEE Access

View full text Add to dashboard Cite

Multiple-choice reading comprehension (MCRC) aims to build an intelligent system that automatically selects an answer from a candidate set when given a passage and a question. Existing MCRC systems rarely consider incorporating external knowledge such as explicit semantic information. In this work, we propose a Contextual and Semantic Fusion Network (CSFN) which effectively integrates contextual and semantic representation. CSFN introduces explicit structured semantics from pre-trained semantic role labeling. Specially, we regard explicit semantic representation as an important feature to fuse with contextual representation, which enriches the representation of sentences. By combining with the transfer learning strategy, the CSFN model has better generalization over limited datasets. To evaluate the ability of our model, we conduct experiments on three MCRC benchmark datasets: RACE, DREAM, and MCTest. Experimental results demonstrate the effectiveness of our proposed model.

show abstract

“…To better analyze the reason why our proposed DHC can be successful, we first validate our model by ablation to investigate the effect of bi-direction matching. Inspired by [37] and [6], we also perform experiments be removing one directional matching on the BERTBASE model at a time. Specifically, we conducted two ablation settings, including uni-directional Question-to-Passage and uni-directional Passage-to-Question, Notes: P-to-Q represents Passage-to-Question, Q-to-P represents Question-to-Passage.…”

Section: Effect Of Bi-direction Matchingmentioning

confidence: 99%

Dual Head-wise Coattention Network for Machine Comprehension with Multiple-Choice Questions

Liu

Huang

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Multiple-choice Machine Comprehension (MC) is an important and challenging nature language processing (NLP) task where the machine (model) is required to make the best answer from candidates given particular passage and question. Existing approaches either only utilize the powerful pre-trained language models or only rely on an over complicated matching network that can capture relationship among a triplet of question, passage and candidate answers effectively. In this paper, we present a novel architecture, Dual Head-wise Coattention network (called DHC), which is a simple and efficient attention neural network designed to perform multiple-choice MC task. Our proposed DHC not only support a powerful pre-trained language model as encoder, but also models the MC relationship as attention mechanism straightforwardly, by head-wise matching and aggregating method on multiple layers, which better model relationships sufficiently between question and passage, and cooperate with large pre-trained language models more efficiently. To evaluate the performance, we evaluate our proposed model on five challenging and well-known datasets for multiple-choice M-C: RACE, DREAM, SemEval-2018 Task 11, OpenBookQA, and TOEFL. Extensive experimental results demonstrate that our proposal can achieve a significant increase in accuracy comparing existing models based on all five datasets, and it consistently outperforms all tested baselines including the state-of-the-arts techniques. More remarkably, our proposal is a pluggable and more flexible model, and it thus can be plugged into any pre-trained Language Models based on BERT. Ablation studies demonstrate its state-of-the-art (SOTA) performance and generalization.

show abstract

DCMN+: Dual Co-Matching Network for Multi-Choice Reading Comprehension

Cited by 105 publications

References 21 publications

SG-Net: Syntax-Guided Machine Reading Comprehension

SG-Net: Syntax-Guided Machine Reading Comprehension

Contextual and Semantic Fusion Network for Multiple-Choice Reading Comprehension

Dual Head-wise Coattention Network for Machine Comprehension with Multiple-Choice Questions

Contact Info

Product

Resources

About