Comparing Word Representations for Implicit Discourse Relation Classification

Abstract

This paper presents a detailed comparative framework for assessing the usefulness of unsupervised word representations for identifying so-called implicit discourse relations. Specifically, we compare standard one-hot word pair representations against low-dimensional ones based on Brown clusters and word embeddings. We also consider various word vector combination schemes for deriving discourse segment representations from word vectors, and compare representations based either on all words or limited to head words. Our main finding is that denser representations systematically outperform sparser ones and give state-of-the-art performance or above without the need for additional hand-crafted features.

Multi-task Attention-based Neural Networks for Implicit Discourse Relationship Representation and Identification

“…With the release of PDTB 2.0 (Prasad et al, 2008), lots of work has been done for discourse relation identification on natural (i.e., genuine) discourse data (Pitler et al, 2009;Lin et al, 2009;Wang et al, 2010;Zhou et al, 2010;Braud and Denis, 2015;Fisher and Simmons, 2015) with the use of traditional NLP linguistically informed features and machine learning algorithms. Recently, more and more researchers resorted to neural networks for implicit discourse recognition (Zhang et al, 2015;Qin et al, 2016a;Liu and Li, 2016;Braud and Denis, 2016;Wu et al, 2016).…”

A Systematic Study of Neural Discourse Models for Implicit Discourse Relation

“…Max pooling is known to be very effective in vision, but it is unclear what pooling function works well when it comes to pooling word vectors. Summation pooling and mean pooling have been claimed to perform well at composing meaning of a short phrase from individual word vectors (Le and Mikolov, 2014;Blacoe and Lapata, 2012;Mikolov et al, 2013b;Braud and Denis, 2015). The Arg1 vector a 1 and Arg2 vector a 2 are computed by applying element-wise pooling function f on all of the N 1 word vectors in Arg1 w 1 1:N 1 and all of the N 2 word vectors in Arg2 w 2 1:N 2 respectively:…”

Do We Need Cross Validation for Discourse Relation Classification?

“…Recently the distributed word representations (Bengio et al, 2003;Mikolov et al, 2013) have shown an advantage in dealing with data sparsity problem (Braud and Denis, 2015). Many deep learning methods have been proved to be helpful in discourse relation parsing and achieved some significant progresses.…”