Global Optimization under Length Constraint for Neural Text Summarization

Makino, Takuya; Iwakura, Tomoya; Takamura, Hiroya; Okumura, Manabu

doi:10.18653/v1/p19-1099

Cited by 33 publications

(16 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we present a two-stage strategy to over-generate, then score system summaries externally for faithfulness and overall quality. Previous work has sought to control various aspects of the generated summary, including the style, length and amount of reused text (Kikuchi et al, 2016;Hu et al, 2017;Fan et al, 2018;Makino et al, 2019;Song et al, 2020). In contrast, our generator focuses on producing multiple variants of the target summary that have diverse content and varying lengths.…”

Section: Related Workmentioning

confidence: 99%

A New Approach to Overgenerating and Scoring Abstractive Summaries

Song¹,

Wang²,

Feng³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

We propose a new approach to generate multiple variants of the target summary with diverse content and varying lengths, then score and select admissible ones according to users' needs. Abstractive summarizers trained on single reference summaries may struggle to produce outputs that achieve multiple desirable properties, i.e., capturing the most important information, being faithful to the original, grammatical and fluent. In this paper, we propose a two-staged strategy to generate a diverse set of candidate summaries from the source text in stage one, then score and select admissible ones in stage two. Importantly, our generator gives a precise control over the length of the summary, which is especially well-suited when space is limited. Our selectors are designed to predict the optimal summary length and put special emphasis on faithfulness to the original text. Both stages can be effectively trained, optimized and evaluated. Our experiments on benchmark summarization datasets suggest that this paradigm can achieve state-of-the-art performance.Source Text: A court here Thursday sentenced a 24-year-old man to 10 years in jail after he admitted pummelling his baby son to death to silence him while watching television. Left to Right Generation (1 Summary) Confidence Driven Generation (4 Summaries) Man who Man gets 10 years Man who killed [. . . ] Man who kill the baby gets 10 years Man who killed baby to hear television better gets 10 Man who kill the baby to hear television gets 10 years Man who killed baby to hear television better gets 10 years Man who kill the baby to hear television better gets 10 years

show abstract

Section: Related Workmentioning

confidence: 99%

A New Approach to Overgenerating and Scoring Abstractive Summaries

Song¹,

Wang²,

Feng³

et al. 2021

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Langua

View full text Add to dashboard Cite

show abstract

“…Furthermore, unlike most previous methods that only control a single binary attribute (e.g., positive and negative sentiments), our approach can further control multiple fine-grained attributes such as sentence length and the existence of specific words. Note that controlling such fine-grained attributes has already been studied in the previous works for other tasks (Post and Vilar 2018;Makino et al 2019), which only serves as a case study to demonstrate the generality of our method.…”

Section: Related Workmentioning

confidence: 99%

Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning

Liu

Zhang

et al. 2020

AAAI

View full text Add to dashboard Cite

Typical methods for unsupervised text style transfer often rely on two key ingredients: 1) seeking the explicit disentanglement of the content and the attributes, and 2) troublesome adversarial learning. In this paper, we show that neither of these components is indispensable. We propose a new framework that utilizes the gradients to revise the sentence in a continuous space during inference to achieve text style transfer. Our method consists of three key components: a variational auto-encoder (VAE), some attribute predictors (one for each attribute), and a content predictor. The VAE and the two types of predictors enable us to perform gradient-based optimization in the continuous space, which is mapped from sentences in a discrete space, to find the representation of a target sentence with the desired attributes and preserved content. Moreover, the proposed method naturally has the ability to simultaneously manipulate multiple fine-grained attributes, such as sentence length and the presence of specific words, when performing text style transfer tasks. Compared with previous adversarial learning based methods, the proposed method is more interpretable, controllable and easier to train. Extensive experimental studies on three popular text style transfer tasks show that the proposed method significantly outperforms five state-of-the-art methods.

show abstract

“…More recently, Takase and Okazaki (2019) modified the positional encoding from the Transformer (Vaswani et al, 2017) to encode allowable lengths. Makino et al (2019) proposed a loss function that encourages summaries within desired lengths. Saito et al (2020) introduced a model that controls both output length and informativeness.…”

Section: Length-controlled Text Generationmentioning

confidence: 99%

Adapting End-to-End Speech Recognition for Readable Subtitles

Liu¹,

Niehues²,

Spanakis³

2020

Proceedings of the 17th International Conference on Spoken Language Translation

View full text Add to dashboard Cite

Automatic speech recognition (ASR) systems are primarily evaluated on transcription accuracy. However, in some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. Therefore, this work focuses on ASR with output compression, a task challenging for supervised approaches due to the scarcity of training data. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. We then compare several methods of end-to-end speech recognition under output length constraints. The experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities. Furthermore, the best performance in terms of WER and ROUGE scores is achieved by explicitly modeling the length constraints within the end-to-end ASR system.

show abstract

Global Optimization under Length Constraint for Neural Text Summarization

Cited by 33 publications

References 24 publications

A New Approach to Overgenerating and Scoring Abstractive Summaries

A New Approach to Overgenerating and Scoring Abstractive Summaries

Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning

Adapting End-to-End Speech Recognition for Readable Subtitles

Contact Info

Product

Resources

About