Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification

Abstract

We present a novel multi-task attentionbased neural network model to address implicit discourse relationship representation and identification through two types of representation learning, an attentionbased neural network for learning discourse relationship representation with two arguments and a multi-task framework for learning knowledge from annotated and unannotated corpora. The extensive experiments have been performed on two benchmark corpora (i.e., PDTB and CoNLL-2016 datasets). Experimental results show that our proposed model outperforms the state-of-the-art systems on benchmark corpora.

Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph

“…Early studies (Pitler et al, 2008;Lin et al, 2009Lin et al, , 2014 focused on extracting linguistic and semantic features from two discourse units. Recent research (Zhang et al, 2015;Ji and Eisenstein, 2015;Ji et al, 2016) tried to model compositional meanings of two discourse units by exploiting interactions between words in two units with more and more complicated neural network models, including the ones using neural tensor (Chen et al, 2016;Qin et al, 2016;Lei et al, 2017) and attention mechanisms Lan et al, 2017;). Another trend is to alleviate the shortage of annotated data by leveraging related external data, such as explicit discourse relations in PDTB Lan et al, 2017;Qin et al, 2017) and unlabeled data obtained elsewhere Lan et al, 2017), often in a multi-task joint learning framework.…”

Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition

“…Qin et al 2017;Lan et al 2017;Dai and Huang 2018;Lei et al 2018), our model still achieves F1 improvements of 1.53% on Comp. and 7.2% on Temp., the numbers of samples belonging to which are the two least in all classes as shown inTable 2.…”

Learning to Explicitate Connectives with Seq2Seq Network for Implicit Discourse Relation Classification

“…Recently, neural networks have shown an advantage of dealing with data sparsity problem, and many deep learning methods have been proposed for discourse parsing, including convolutional (Zhang et al, 2015), recurrent (Ji et al, 2016), character-based (Qin et al, 2016a), adversarial (Qin et al, 2017) neural networks, and pair-aware neural sentence modeling (Cai and Zhao, 2017). Multi-task learning has also been shown to be beneficial on this task (Lan et al, 2017).…”

“…h e andh d are then combined using a linear layer (Lan et al, 2017). As illustrated in Equation 11, the linear layer acts as a gate to determine how much information from the sequence-to-sequence network should be mixed into the original sentence's representations from the encoder.…”