The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task

Abstract

This paper describes the submission of the Ni-uTrans end-to-end speech translation system for the IWSLT 2021 offline task, which translates from the English audio to German text directly without intermediate transcription. We use the Transformer-based model architecture and enhance it by Conformer, relative position encoding, and stacked acoustic and textual encoding. To augment the training data, the English transcriptions are translated to German translations. Finally, we employ ensemble decoding to integrate the predictions from several models trained with the different datasets. Combining these techniques, we achieve 33.84 BLEU points on the MuST-C En-De test set, which shows the enormous potential of the end-to-end model.

Does Simultaneous Speech Translation need Simultaneous Models?

“…Two main word detection strategies are currently employed by the community: fixed (Ma et al, 2020b), and adaptive (Ma et al, 2020b;Ren et al, 2020;Zeng et al, 2021;Chen et al, 2021). The fixed word detection strategy represents the easiest way to address the problem since it assumes that a fixed amount of time is required to pronounce every word, disregarding the information contained in the audio.…”

“…On the contrary, the adaptive word detection strategy determines the number of words by looking at the content of the audio. The decision about waiting or emitting can be taken through an Automatic Speech Recognition decoder (Chen et al, 2021) 2 or through a Connectionist Temporal Classification (Graves et al, 2006) -or CTC -module (Ren et al, 2020;Zeng et al, 2021), responsible for directly detecting the number of words every time a speech chunk is received by the system.…”

“…The most popular decision policy is the wait-k, a straightforward heuristic that prescribes to wait for a predefined number of words before starting to generate the translation. Initially proposed by Ma et al (2020b) for simultaneous machine translation (SimulMT), the wait-k has been widely adopted in SimulST (Ma et al, 2020b;Ren et al, 2020;Chen et al, 2021;Zeng et al, 2021; thanks to its simplicity. Apart from wait-k, other attempts have been made to develop decision policies learned by the SimulST system itself (Ma et al, 2019b;Zaidi et al, 2021;Liu et al, 2021a,b), all resulting in computationally expensive models with limited diffusion.…”