The DISRPT 2021 Shared Task on Elementary Discourse Unit Segmentation, Connective Detection, and Relation Classification

Zeldes, Amir; Liu, Yang Janet; Iruskieta, Mikel; Muller, Philippe; Braud, Chloé; Badene, Sonia

doi:10.18653/v1/2021.disrpt-1.1

Cited by 8 publications

(9 citation statements)

References 29 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The latter, meanwhile, connects the EDU-boundary representations, which enhances the model's ability to capture long-range dependencies between EDUs. 2 We could not compare our segmentation results with the DISRPT 2019 Shared Task (Zeldes et al, 2019) participants. We found few inconsistencies in the settings.…”

Section: Ablation Studymentioning

confidence: 99%

RST Parsing from Scratch

Nguyen,

Joty

et al. 2021

Preprint

View full text Add to dashboard Cite

We introduce a novel top-down end-to-end formulation of document level discourse parsing in the Rhetorical Structure Theory (RST) framework. In this formulation, we consider discourse parsing as a sequence of splitting decisions at token boundaries and use a seq2seq network to model the splitting decisions. Our framework facilitates discourse parsing from scratch without requiring discourse segmentation as a prerequisite; rather, it yields segmentation as part of the parsing process. Our unified parsing model adopts a beam search to decode the best tree structure by searching through a space of high scoring trees. With extensive experiments on the standard English RST discourse treebank, we demonstrate that our parser outperforms existing methods by a good margin in both end-to-end parsing and parsing with gold segmentation. More importantly, it does so without using any handcrafted features, making it faster and easily adaptable to new languages and domains.

show abstract

Section: Ablation Studymentioning

confidence: 99%

RST Parsing from Scratch

Nguyen,

Joty

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Computational analysis of discourse has been the focus of several shared tasks (Xue et al, , 2016Zeldes et al, 2019Zeldes et al, , 2021, and there have been several discourse-annotated corpora for multiple languages (Zeyrek and Webber, 2008;Meyer et al, 2011;Danlos et al, 2012;Zhou and Xue, 2015;Zeyrek et al, 2020;da Cunha et al, 2011;Das and Stede, 2018;Afantenos et al, 2012). Despite their widespread use, implicit sense classification remains a challenging task (Liang et al, 2020), and discourse models have been shown not to perform well under even gradual domain shift (Atwell et al, 2021), which may be the result of the limited timeframe and distribution of the articles contained in the most commonly used English discourse datasets, the Penn Discourse Treebank (Miltsakaki et al, 2004;Prasad et al, 2008;Webber et al, 2019) and the RST Discourse Treebank (RST-DT) (Carlson et al, 2001).…”

Section: Discourse and Domain Shiftmentioning

confidence: 99%

The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error

Atwell¹,

Sicilia²,

Hwang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Discourse analysis allows us to attain inferences of a text document that extend beyond the sentence-level. The current performance of discourse models is very low on texts outside of the training distribution's coverage, diminishing the practical utility of existing models. There is need for a measure that can inform us to what extent our model generalizes from the training to the test sample when these samples may be drawn from distinct distributions. While this can be estimated via distribution shift, we argue that this does not directly correlate with change in the observed error of a classifier (i.e. error-gap). Thus, we propose to use a statistic from the theoretical domain adaptation literature which can be directly tied to error-gap. We study the bias of this statistic as an estimator of error-gap both theoretically and through a large-scale empirical study of over 2400 experiments on 6 discourse datasets from domains including, but not limited to: news, biomedical texts, TED talks, Reddit posts, and fiction. Our results not only motivate our proposal and help us to understand its limitations, but also provide insight on the properties of discourse models and datasets which improve performance in domain adaptation. For instance, we find that non-news datasets are slightly easier to transfer to than news datasets when the training and test sets are very different. Our code and an associated Python package are available to allow practitioners to make more informed model and dataset choices. 1

show abstract

“…Outside the PDTB-3 framework, intra-sentential discourse relations are handled by (1) identifying discourse units (DUs), (2) attaching them to one another, and (3) associating the attachment with a coherence relation (Muller et al, 2012). One can therefore ask why we did not simply adopt this framework in the PDTB-3 and exploit the relatively good performance by systems in the DISRPT shared task on sentence-level discourse unit segmentation (Zeldes et al, 2019). There are two main reasons: First, DISRPT (and the approaches to discourse structure it covers) assumes that discourse segments cover a sentence with a non-overlapping partition.…”

Section: Related Workmentioning

confidence: 99%

“…Of course, there are "work-arounds" for over-segmentation, such as RST's use of a SAME-SEGMENT relation (Mann and Thompson, 1988), and under-segmentation can be addressed through additional segmentation. However, we decided that starting from scratch would allow us to clearly identify the problems of parsing intra-sentential implicits, at which point, we could consider what we could adopt from work done on the DISRPT shared task on sentence-level discourse unit segmentation (Zeldes et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

“…The task is considered to be important for downstream tasks such as question answering (Jansen et al, 2014), machine translation (Li et al, 2014), and text summarization (Cohan et al, 2018). There are various approaches to discourse parsing, corresponding to different views of (1) what constitutes the segments of discourse, (2) what structures can be built from such segments, and (3) what semantic and/or rhetorical relations can hold between such segments (Xue et al, 2015;Zeldes et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Revisiting Shallow Discourse Parsing in the PDTB-3: Handling Intra-sentential Implicits

Zhang¹,

Webber²

2022

Preprint

View full text Add to dashboard Cite

In the PDTB-3, several thousand implicit discourse relations were newly annotated within individual sentences, adding to the over 15,000 implicit relations annotated across adjacent sentences in the PDTB-2. Given that the position of the arguments to these intra-sentential implicits is no longer as well-defined as with inter-sentential implicits, a discourse parser must identify both their location and their sense. That is the focus of the current work. The paper provides a comprehensive analysis of our results, showcasing model performance under different scenarios, pointing out limitations and noting future directions.

show abstract

The DISRPT 2021 Shared Task on Elementary Discourse Unit Segmentation, Connective Detection, and Relation Classification

Cited by 8 publications

References 29 publications

RST Parsing from Scratch

RST Parsing from Scratch

The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error

Revisiting Shallow Discourse Parsing in the PDTB-3: Handling Intra-sentential Implicits

Contact Info

Product

Resources

About