A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards

Dou, Zi-Yi; Kumar, Sachin; Tsvetkov, Yulia

doi:10.18653/v1/2020.ngt-1.7

Cited by 9 publications

(11 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventional cross-lingual summarization methods mainly focus on incorporating bilingual information into the pipeline methods (Leuski et al, 2003;Ouyang et al, 2019;Orȃsan and Chiorean, 2008;Wan et al, 2010;Wan, 2011;Yao et al, 2015;Zhang et al, 2016b), i.e., translation and then summarization or summarization and then translation. Due to the difficulty of acquiring cross-lingual summarization dataset, some previous researches focus on constructing datasets (Ladhak et al, 2020;Scialom et al, 2020;Yela-Bello et al, 2021;Zhu et al, 2019;Hasan et al, 2021;Perez-Beltrachini and Lapata, 2021;Varab and Schluter, 2021), mixed-lingual pre-training (Xu et al, 2020), knowledge distillation (Nguyen and Tuan, 2021), contrastive learning (Wang et al, 2021) or zero-shot approaches (Ayana et al, 2018;Duan et al, 2019;Dou et al, 2020), i.e., using machine translation (MT) or monolingual summarization (MS) or both to train the CLS system. Among them, Zhu et al (2019) propose to use roundtrip translation strategy to obtain large-scale CLS datasets and then present two multi-task learning methods for CLS.…”

Section: Related Workmentioning

confidence: 99%

A Variational Hierarchical Model for Neural Cross-Lingual Summarization

Liang¹,

Meng²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

The goal of the cross-lingual summarization (CLS) is to convert a document in one language (e.g., English) to a summary in another one (e.g., Chinese). Essentially, the CLS task is the combination of machine translation (MT) and monolingual summarization (MS), and thus there exists the hierarchical relationship between MT&MS and CLS. Existing studies on CLS mainly focus on utilizing pipeline methods or jointly training an end-toend model through an auxiliary MT or MS objective. However, it is very challenging for the model to directly conduct CLS as it requires both the abilities to translate and summarize. To address this issue, we propose a hierarchical model for the CLS task, based on the conditional variational auto-encoder. The hierarchical model contains two kinds of latent variables at the local and global levels, respectively. At the local level, there are two latent variables, one for translation and the other for summarization. As for the global level, there is another latent variable for cross-lingual summarization conditioned on the two local-level variables. Experiments on two language directions (English⇔Chinese) verify the effectiveness and superiority of the proposed approach. In addition, we show that our model is able to generate better cross-lingual summaries than comparison models in the few-shot setting.

show abstract

Section: Related Workmentioning

confidence: 99%

A Variational Hierarchical Model for Neural Cross-Lingual Summarization

Liang¹,

Meng²,

Zhou³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To solve this problem, propose a mix-lingual XLS model which is pre-trained with MLM, DAE, MS, TSC and MT tasks 17 . Dou et al (2020) utilize the XLS, MT and MS to pretrain the XLS model. Wang et al (2022b) focus on dialogue-oriented XLS and extend mBART-50 with AcI, UP, MS and MT tasks via the second pretraining stage.…”

Section: Pre-training Frameworkmentioning

confidence: 99%

“…Wang et al (2022b) focus on dialogue-oriented XLS and extend mBART-50 with AcI, UP, MS and MT tasks via the second pretraining stage. Note that , Dou et al (2020) and Wang et al (2022b) only focus on XLS task. Furthermore, ∆LM (Ma et al, 2021) and mT6 (Chi et al, 2021a) are presented towards general cross-lingual abilities.…”

Section: Pre-training Frameworkmentioning

confidence: 99%

“…In detail, the pipeline models adopt either translate-then-summarize approaches (Leuski et al, 2003;Boudin et al, 2011;Wan, 2011;Yao et al, 2015;Zhang et al, 2016;Linhares Pontes et al, 2018;Wan et al, 2018;Ouyang et al, 2019) or summarize-then-translate approaches (Orȃsan and Chiorean, 2008;Wan et al, 2010). For end-to-end models, they mainly fall into four categories, i.e., multi-task methods (Zhu et al, 2019;Takase and Okazaki, 2020;Cao et al, 2020;Bai et al, 2021a;Liang et al, 2022), knowledge-distillation methods (Ayana et al, 2018;Duan et al, 2019;Nguyen and Tuan, 2021), resource-enhanced methods Jiang et al, 2022) and pre-training methods (Dou et al, 2020;Ma et al, 2021;Chi et al, 2021a;Wang et al, 2022b). For each category, we will thoroughly go through the previous work and discuss the corresponding pros and cons.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Cross-Lingual Summarization

Wang¹,

Meng²,

Zheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Cross-lingual summarization is the task of generating a summary in one language (e.g., English) for the given document(s) in a different language (e.g., Chinese). Under the globalization background, this task has attracted increasing attention of the computational linguistics community. Nevertheless, there still remains a lack of comprehensive review for this task. Therefore, we present the first systematic critical review on the datasets, approaches and challenges in this field. Specifically, we carefully organize existing datasets and approaches according to different construction methods and solution paradigms, respectively. For each type of datasets or approaches, we thoroughly introduce and summarize previous efforts and further compare them with each other to provide deeper analyses. In the end, we also discuss promising directions and offer our thoughts to facilitate future research. This survey is for both beginners and experts in cross-lingual summarization, and we hope it will serve as a starting point as well as a source of new ideas for researchers and engineers interested in this area.

show abstract

“…With the widespread use of deep neural networks, many Transformer-based end-to-end methods [4], [5], [8], [9], [36], [48], [49] are proposed to directly understand bilingual semantics and avoid the error propagation problem. Due to lacking large-scale CLS datasets, Duan et al [9] and Dou et al [8] explore training end-to-end models with zeroshot learning.…”

Section: Introductionmentioning

confidence: 99%

ClueGraphSum: Let Key Clues Guide the Cross-Lingual Abstractive Summarization

Jiang¹,

Tu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Cross-Lingual Summarization (CLS) is the task to generate a summary in one language for an article in a different language. Previous studies on CLS mainly take pipeline methods or train the end-to-end model using the translated parallel data. However, the quality of generated cross-lingual summaries needs more further efforts to improve, and the model performance has never been evaluated on the hand-written CLS dataset. Therefore, we first propose a clue-guided cross-lingual abstractive summarization method to improve the quality of cross-lingual summaries, and then construct a novel hand-written CLS dataset for evaluation. Specifically, we extract keywords, named entities, etc. of the input article as key clues for summarization and then design a clue-guided algorithm to transform an article into a graph with less noisy sentences. One Graph encoder is built to learn sentence semantics and article structures and one Clue encoder is built to encode and translate key clues, ensuring the information of important parts are reserved in the generated summary. These two encoders are connected by one decoder to directly learn cross-lingual semantics. Experimental results show that our method has stronger robustness for longer inputs and substantially improves the performance over the strong baseline, achieving an improvement of 8.55 ROUGE-1 (English-to-Chinese summarization) and 2.13 MoverScore (Chinese-to-English summarization) scores over the existing SOTA.

show abstract

A Deep Reinforced Model for Zero-Shot Cross-Lingual Summarization with Bilingual Semantic Similarity Rewards

Cited by 9 publications

References 19 publications

A Variational Hierarchical Model for Neural Cross-Lingual Summarization

A Variational Hierarchical Model for Neural Cross-Lingual Summarization

A Survey on Cross-Lingual Summarization

ClueGraphSum: Let Key Clues Guide the Cross-Lingual Abstractive Summarization

Contact Info

Product

Resources

About