Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoff

Abstract

Blockwise self-attentional encoder models have recently emerged as one promising end-to-end approach to simultaneous speech translation. These models employ a blockwise beam search with hypothesis reliability scoring to determine when to wait for more input speech before translating further. However, this method maintains multiple hypotheses until the entire speech input is consumed -this scheme cannot directly show a single incremental translation to users. Further, this method lacks mechanisms for controlling the quality vs. latency tradeoff. We propose a modified incremental blockwise beam search incorporating local agreement or hold-n policies for qualitylatency control. We apply our framework to models trained for online or offline translation and demonstrate that both types can be effectively used in online mode.Experimental results on MuST-C show 0.6-3.6 BLEU improvement without changing latency or 0.8-1.4 s latency improvement without changing quality.

CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022

“…Therefore, we proposed an improved version of the AL metric, which was later independently proposed under name length-adaptive average lagging (LAAL; Papi et al, 2022). To remedy the over-generation problem, we proposed an improved version of the beam search algorithm in Polák et al (2023b). While this led to significant improvements in the quality-latency tradeoff, the decoding still relied on label-synchronous decoding.…”