Neural Generative Rhetorical Structure Parsing

Abstract

Rhetorical structure trees have been shown to be useful for several document-level tasks including summarization and document classification. Previous approaches to RST parsing have used discriminative models; however, these are less sample efficient than generative models, and RST parsing datasets are typically small. In this paper, we present the first generative model for RST parsing. Our model is a document-level RNN grammar (RNNG) with a bottom-up traversal order. We show that, for our parser's traversal order, previous beam search algorithms for RNNGs have a left-branching bias which is ill-suited for RST parsing. We develop a novel beam search algorithm that keeps track of both structureand word-generating actions without exhibiting this branching bias and results in absolute improvements of 6.8 and 2.9 on unlabelled and labelled F1 over previous algorithms. Overall, our generative model outperforms a discriminative model with the same features by 2.6 F1 points and achieves performance comparable to the state-of-the-art, outperforming all published parsers from a recent replication study that do not use additional training data. ✠ JUSTIFY Acme Inc. has closed several widget factories. | ATTRIBUTION The CEO told investors they were no longer profitable.

MEGA RST Discourse Treebanks with Structure and Nuclearity from Scalable Distant Sentiment Supervision

“…10) already achieves good performance, with Dyer et al (2016) delivering promising results by using greedy decoding. As a recent example for discourse parsing, Mabona et al (2019) successfully combine standard beam-search with shift-reduce parsing using two parallel beams for shift and reduce actions. Overall, recent work shows that beam-search approaches and their possible extensions can effectively address scalability issues in multiple parsing scenarios.…”

“…Inspired by the recent success in applying beamsearch to enhance the scalability of multiple NLP parsing tasks (Mabona et al, 2019;Fried et al, 2017;Dyer et al, 2016;Vinyals et al, 2015), we propose a novel heuristic beam-search approach that can automatically generate discourse trees containing structure-and nuclearity-attributes for documents of arbitrary length.…”

“…In this work, we propose a significant extension to this line of research by introducing a scalable solution for documents of arbitrary length and further moving beyond just predicting the tree-structure by incorporating the nuclearity attribute, oftentimes critical in informing downstream tasks (Marcu, 2000;Ji and Smith, 2017;Shiv and Quirk, 2019). Inspired by the recent success of heuristic search in NLP tasks involving trees (e.g., Fried et al (2017); Mabona et al (2019)), we develop a beam-search strategy implementing an exploration-exploitation trade-off, as commonly used in reinforcementlearning (RL) (Poole and Mackworth, 2010).…”

Max-Margin Incremental CCG Parsing

“…There has also been some recent work on reducing the imbalanced probability bias. Mabona et al (2019) propose an algorithmic solution for organising beam search into buckets that have the same number of expensive transitions. Crabbé et al (2019) propose a sampling based approach with the same motivation of controlling which hypotheses are being compared.…”

Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining

“…The combination of these two limitations has been one of the main reasons for the limited application of neural discourse parsing for more diverse downstream tasks. While there have been neural discourse parsers proposed (Braud et al, 2017;Yu et al, 2018;Mabona et al, 2019), they still cannot consistently outperform traditional approaches when applied to the RST-DT dataset, where the amount of training data is arguably insufficient for such data-intensive approaches.…”