Low-Latency Neural Speech Translation

Abstract

Through the development of neural machine translation, the quality of machine translation systems has been improved significantly. By exploiting advancements in deep learning, systems are now able to better approximate the complex mapping from source sentences to target sentences. But with this ability, new challenges also arise. An example is the translation of partial sentences in low-latency speech translation. Since the model has only seen complete sentences in training, it will always try to generate a complete sentence, though the input may only be a partial sentence. We show that NMT systems can be adapted to scenarios where no task-specific training data is available. Furthermore, this is possible without losing performance on the original training data. We achieve this by creating artificial data and by using multi-task learning. After adaptation, we are able to reduce the number of corrections displayed during incremental output construction by 45%, without a decrease in translation quality.

Re-Translation Strategies for Long Form, Simultaneous, Spoken Language Translation

“…Following Niehues et al (2016Niehues et al ( , 2018 erasure, which measures the length of the suffix that is deleted to produce the next revision. Let o i be the i th output of a PTL.…”

“…Standard models trained on full sentences are unlikely to perform well when applied to prefixes. We alleviate this problem by generating prefix pairs from our parallel training corpus, and subsequently training on a 1:1 mix of full-sentence and prefix pairs (Niehues et al, 2018;Dalvi et al, 2018). Following Niehues et al (2018), we augment our training data with prefix pairs created by selecting a source prefix length uniformly at random, then selecting a target length either proportionally according to sentence length, or based on self-contained word alignments.…”

Re-translation versus Streaming for Simultaneous Translation

“…Re-translation was originally investigated by Niehues et al (2016Niehues et al ( , 2018, and more recently extended by Arivazhagan et al (2019a), who propose a suitable evaluation framework, and use it to assess inference-time re-translation strategies for speech translation. We adopt their inference-time heuristics to stabilize re-translation, and extend them with prefix training from Niehues et al (2018). Where they experiment on TED talks, compare only to vanilla re-translation and use proprietary NMT, we follow recent work on streaming by using WMT training and test data, and provide a novel comparison to streaming approaches.…”

ELITR Non-Native Speech Translation at IWSLT 2020

“…OPUS-A is a Transformer Base model trained on 1M sentence pairs each for 7 European target languages: Czech, Dutch, French, German, Hungarian, Polish, and Romanian. OPUS-B is a Transformer Big model trained on a total of 231M sentence pairs covering 41 target languages that are of particular interest to the project 5 After initial training, OPUS-B was finetuned on an augmented version of the dataset that includes partial sentence pairs, artificially generated by truncating the original sentence pairs (similar to Niehues et al, 2018). We produce up to 10 truncated sentence pairs for every one original pair.…”