Re-Translation Strategies for Long Form, Simultaneous, Spoken Language Translation

Abstract

There has been great progress in improving streaming machine translation, a simultaneous paradigm where the system appends to a growing hypothesis as more source content becomes available. We study a related problem in which revisions to the hypothesis beyond strictly appending words are permitted. This is suitable for applications such as live captioning an audio feed. In this setting, we compare custom streaming approaches to re-translation, a straightforward strategy where each new source token triggers a distinct translation from scratch. We find retranslation to be as good or better than stateof-the-art streaming systems, even when operating under constraints that allow very few revisions. We attribute much of this success to a previously proposed data-augmentation technique that adds prefix-pairs to the training data, which alongside wait-k inference forms a strong baseline for streaming translation. We also highlight re-translation's ability to wrap arbitrarily powerful MT systems with an experiment showing large improvements from an upgrade to its base model.

Beyond Sentence-Level End-to-End Speech Translation: Context Helps

“…Context-aware ST extends the sentence-level ST towards streaming ST which allows models to access unlimited previous audio inputs. Instead of improving contextual modeling, many studies on streaming ST aim at developing better sentence/word segmentation policies to avoid segmentation errors that greatly hurt translation (Matusov et al, 2007;Rangarajan Sridhar et al, 2013;Iranzo-Sánchez et al, 2020;Arivazhagan et al, 2020b). Very recently, Ma et al (2020b) proposed a memory augmented Transformer encoder for streaming ST, where the previous audio features are summarized into a growing continuous memory to improve the model's context awareness.…”

BIT’s system for AutoSimulTrans2021

“…Another type is translation based on dynamic refresh without the need to adjust the machine translation model. Whenever the input increases, translate all input and overwrite the translation result that has been generated last time (Niehues et al, 2016) (Arivazhagan et al, 2020b) (Arivazhagan et al, 2020a).…”

ELITR Non-Native Speech Translation at IWSLT 2020

“…MT Wrapper has a parameter to control the stability and latency. It can mask the last k words of incomplete sentences from the ASR output, as in Ma et al (2019) and Arivazhagan et al (2019), considering only the currently completed sentences, or only the "stable" sentences, which are beyond the ASR and punctuator processing window and never change. We do not tune these parameters in the validation.…”