On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification

Abstract

<jats:p>Implicit discourse relation classification is one of the most difficult parts in shallow discourse parsing as the relation prediction without explicit connectives requires the language understanding at both the text span level and the sentence level. Previous studies mainly focus on the interactions between two arguments. We argue that a powerful contextualized representation module, a bilateral multi-perspective matching module, and a global information fusion module are all important to implicit discourse analysis. We propose a novel model to combine these modules together. Extensive experiments show that our proposed model outperforms BERT and other state-of-the-art systems on the PDTB dataset by around 8% and CoNLL 2016 datasets around 16%. We also analyze the effectiveness of different modules in the implicit discourse relation classification task and demonstrate how different levels of representation learning can affect the results.</jats:p>

A Label Dependence-aware Sequence Generation Model for Multi-level Implicit Discourse Relation Recognition

“…Second, in most cases, jointly inferring multi-level labels (HierMTN-CRF-RoBERTa, OurEncoder+OurDecoder, LDSGM) performs better than separately predicting in BMGF-RoBERTa, Table 2: Comparison with recent models on the consistency among multi-level label predictions. We run the code of BMGF-RoBERTa (Liu et al 2020) and report the results.…”

“…Therefore, we report the label-wise F 1 scores for the second-level labels in Table 4. A closer look into the results reveals that though our LDSGM model outperforms BMGF-RoBERTa (Liu et al 2020) on most majority labels, the F 1 scores for three minority labels are still 0%. Besides, the BERT-based model (Kishimoto, Murawaki, and Kurohashi 2020) that small numbers of training examples are insufficient to optimize the huge amounts of parameters in these models.…”

“…With the rapid development of deep learning, previous IDRR models relying on human-designed features have evolved to neural network based models (Zhang et al 2015;Lei et al 2017;Xu et al 2019;Liu et al 2020;Zhang et al 2021). Despite their success, these studies usually train multiple models to predict multi-level labels independently, while ignoring the dependence between these hierarchically structured labels.…”

RST Discourse Parser for Russian: An Experimental Study of Deep Learning Models

“…For the classification subtasks, we use the BiMPM [31] multi-perspective symmetric matching model, originally proposed for sentence matching and used to extract implicit interactions between text spans [17]. In the BiMPM model, two textual inputs are encoded in the BiLSTM encoder and the generated vectors are matched in both directions on each time step.…”

Exploring Discourse Structures for Argument Impact Classification

“…As claim representations { ← − H i } and { − → H i } from RoBERTa are not bidirectional, we need to combine them and control which of them matters more. The gated fusion (Liu et al, 2020) has been shown of a better mixture than the combination of multihead attention and layer normalization. We use it to maintain the powerful representative features and carry useful historical context information:…”