Attend, Translate and Summarize: An Efficient Method for Neural Cross-Lingual Summarization

Abstract

Cross-lingual summarization aims at summarizing a document in one language (e.g., Chinese) into another language (e.g., English). In this paper, we propose a novel method inspired by the translation pattern in the process of obtaining a cross-lingual summary. We first attend to some words in the source text, then translate them into the target language, and summarize to get the final summary. Specifically, we first employ the encoder-decoder attention distribution to attend to the source words. Second, we present three strategies to acquire the translation probability, which helps obtain the translation candidates for each source word. Finally, each summary word is generated either from the neural distribution or from the translation candidates of source words. Experimental results on Chinese-to-English and English-to-Chinese summarization tasks have shown that our proposed method can significantly outperform the baselines, achieving comparable performance with the state-of-the-art.

Cross-Lingual Abstractive Summarization with Limited Parallel Resources

“…The method proposed by Cao et al (2020) requires not only parallel summaries but also document pairs translated by MT systems. Another method proposed by Zhu et al (2020) requires bilingual lexicons extracted from large parallel MT datasets (2.08M sentence pairs from eight LDC corpora). We choose not to use these models as baselines since comparing MCLAS with them is unfair.…”

“…Cross-lingual summarization (CLS) helps people efficiently grasp salient information from articles in a foreign language. Neural approaches to CLS require large scale datasets containing millions of cross-lingual document-summary pairs (Zhu et al, 2019; Cao et al, 2020;Zhu et al, 2020). However, two challenges arise with these approaches: 1) most languages are low-resource, thereby lacking document-summary paired data; 2) large parallel datasets across different languages for neuralbased CLS are rare and expensive, especially under the current trend of neural networks.…”

Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation

“…CopyNet is also widely used in text summarization (See et al, 2017;Zhu et al, 2020), automatic postediting (Huang et al, 2019), grammar correction (Zhao et al, 2019a) and so on.…”

“…From a high-level perspective, our methods share a similar Transformer-based architecture with Huang et al (2019) and Zhu et al (2020). Huang et al (2019) employed CopyNet to copy from a draft generated by a pre-trained NMT system.…”

“…Huang et al (2019) employed CopyNet to copy from a draft generated by a pre-trained NMT system. Zhu et al (2020) proposed a method that integrates the operation of attending, translating, and summarizing to do cross-lingual summarization. What distinguishes our PDC from other copy-based architectures lies in that the three novel components (Pointer, Disambiguator and Copier) and the selective BPE strategy can make full and effective use of dictionary knowledge.…”