NCLS: Neural Cross-Lingual Summarization

Abstract

Cross-lingual summarization (CLS) is the task to produce a summary in one particular language for a source document in a different language. Existing methods simply divide this task into two steps: summarization and translation, leading to the problem of error propagation. To handle that, we present an end-to-end CLS framework, which we refer to as Neural Cross-Lingual Summarization (NCLS), for the first time. Moreover, we propose to further improve NCLS by incorporating two related tasks, monolingual summarization and machine translation, into the training process of CLS under multi-task learning. Due to the lack of supervised CLS data, we propose a round-trip translation strategy to acquire two high-quality large-scale CLS datasets based on existing monolingual summarization datasets. Experimental results have shown that our NCLS achieves remarkable improvement over traditional pipeline methods on both English-to-Chinese and Chinese-to-English CLS human-corrected test sets. In addition, NCLS with multi-task learning can further significantly improve the quality of generated summaries. We make our dataset and code publicly available here:Rod gray , 94 , had been taken to hospital by ambulance after he cut his head in a fall at his home … Rod gray was taken to ipswich hospital after falling over at home .Rod gray was taken to Ipswich Hospital after falling down at home. MS RTT English Article (Input for MS or CLS)English Reference (Output for MS) Chinese ReferenceRod gray was taken to ipswich hospital after falling over at home .Rod-Grau wurde nach dem Sturz zu Hause ins ipswich-Krankenhaus gebracht.

Multimodal Summarization with Guidance of Multimodal Reference

“…Multimodal summarization has been proposed to extract the most important information from the multimedia information. The most significant difference between multimodal summarization (Mademlis et al 2016;Li et al 2017;2018b;Zhu et al 2018) and text summarization (Zhu et al 2017;Paulus, Xiong, and Socher 2018;Celikyilmaz et al 2018;Li et al 2018c;Zhu et al 2019) lies in whether the input data contains two or more modalities of data. One of the most significant advantages of the task is that it can use the rich information in multimedia data to improve the quality of the final summary.…”

A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards

“…Then, translateỹ tgt back to the source language using a target-to-source MT model and discard the examples with high reconstruction errors, which are measured with ROUGE (Lin, 2004) scores. The details of this step can be found in Zhu et al (2019).…”

“…The pre-training loss we use is a weighted combination of three objectives. Similarly to Zhu et al (2019), we use an XLS pretraining objective and an MT pre-training objective as described below with some simple but effective improvements. We also introduce an additional objective based on distilling knowledge from a monolingual summarization model.…”

“…Traditional approaches to XLS thus follow a pipeline, for example, summarizing the article in the source language followed by translating the summary into the target language or viceversa (Wan et al, 2010;Wan, 2011). Both of these approaches require separately trained summarization and translation models, and suffer from error propagation (Zhu et al, 2019). Prior studies have attempted to train XLS models in an end-to-end fashion, through knowledge distillation from pre-trained machine translation (MT) or monolingual summarization models (Ayana et al, 2018;Duan et al, 2019), but these approaches have been only shown to work for short outputs.…”

“…Prior studies have attempted to train XLS models in an end-to-end fashion, through knowledge distillation from pre-trained machine translation (MT) or monolingual summarization models (Ayana et al, 2018;Duan et al, 2019), but these approaches have been only shown to work for short outputs. Alternatively, Zhu et al (2019) proposed to automatically translate source-language summaries in the training set thereby generating pseudo-reference summaries in the target language. With this parallel dataset of source documents and target summaries , an end-to-end model is trained to simultaneously summarize and translate using a multi-task objective.…”