NAIST English-to-Japanese Simultaneous Translation System for IWSLT 2021 Simultaneous Text-to-text Task

Fukuda, Ryo; Oka, Yui; Kano, Yasumasa; Yano, Yuki; Ko, Yuka; Tokuyama, Hirotaka; Doi, Kosuke; Sakti, Sakriani; Sudoh, Katsuhito; Nakamura, Satoshi

doi:10.18653/v1/2021.iwslt-1.3

Cited by 1 publication

(1 citation statement)

References 15 publications

(13 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We closely follow previous SiMT works (Ren et al 2020;Zhang, Feng, and Li 2021;Fukuda et al 2021;Liu et al 2021a;Zhao et al 2021) to adopt sequence-level knowledge distillation (Kim and Rush 2016) for all systems. Specifically, we train a full-sentence BASE Transformer (Vaswani et al 2017) as the teacher on the original bilingual dataset, then perform beam-search decoding for the source side of the original bilingual data or newly introduced monolingual data to generate the distilled data.…”

Section: Model Trainingmentioning

confidence: 99%

Improving Simultaneous Machine Translation with Monolingual Data

Hexuan

Liu²,

Liu

et al. 2023

AAAI

View full text Add to dashboard Cite

Simultaneous machine translation (SiMT) is usually done via sequence-level knowledge distillation (Seq-KD) from a full-sentence neural machine translation (NMT) model. However, there is still a significant performance gap between NMT and SiMT. In this work, we propose to leverage monolingual data to improve SiMT, which trains a SiMT student on the combination of bilingual data and external monolingual data distilled by Seq-KD. Preliminary experiments on En-Zh and En-Ja news domain corpora demonstrate that monolingual data can significantly improve translation quality (e.g., +3.15 BLEU on En-Zh). Inspired by the behavior of human simultaneous interpreters, we propose a novel monolingual sampling strategy for SiMT, considering both chunk length and monotonicity. Experimental results show that our sampling strategy consistently outperforms the random sampling strategy (and other conventional typical NMT monolingual sampling strategies) by avoiding the key problem of SiMT -- hallucination, and has better scalability. We achieve +0.72 BLEU improvements on average against random sampling on En-Zh and En-Ja. Data and codes can be found at https://github.com/hexuandeng/Mono4SiMT.

show abstract