Does Simultaneous Speech Translation need Simultaneous Models?

Abstract

In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task. To meet the latency constraints posed by the different application scenarios, multiple dedicated SimulST models are usually trained and maintained, generating high computational costs. In this paper, motivated by the increased social and environmental impact caused by these costs, we investigate whether a single model trained offline can serve not only the offline but also the simultaneous task without the need for any additional training or adaptation. Experiments on en→{de, es} indicate that, aside from facilitating the adoption of well-established offline techniques and architectures without affecting latency, the offline solution achieves similar or better translation quality compared to the same model trained in simultaneous settings, as well as being competitive with the SimulST state of the art.

Efficient yet Competitive Speech Translation: FBK@IWSLT2022

“…In light of the recent work that questions the necessity of a dedicated training procedure for simultaneous model (Papi et al, 2022), we participate in the Simultaneous task with the same model used for the Offline task. Their finding is perfectly aligned with the spirit of this submission toward the reduction of training computational costs.…”

“…We determine when to start generating the output translation adopting the wait-k strategy (Ma et al, 2019) that simply prescribes to wait for k words before starting to generate the translation, where k is a hyper-parameter controlled by the user that can be increased or decreased to directly control the latency of the system. The number of words in a given input speech is determined with an adaptive word detection strategy (Ren et al, 2020), because of its superiority over the fixed strategy in strong models trained in high-resource data conditions (Papi et al, 2022). Our adaptive word detection mechanism exploits the predicted output of CTC module in the encoder (Ren et al, 2020;Zeng et al, 2021) to count the number of words in the source speech.…”

“…In the same vein of reducing the overall training computational costs, we participated also in the simultaneous task using our best offline model and without performing any additional training do adapt it to the simultaneous scenario (Papi et al, 2022). The simultaneous version of our offlinetrained model is realized by applying the wait-k strategy (Ma et al, 2019) with adaptive word detection from the audio input (Ren et al, 2020) that determines the number of words in a speech segment using the greedy prediction of the CTC.…”