Lecture Translator - Speech translation framework for simultaneous lecture translation

Abstract

Foreign students at German universities often have difficulties following lectures as they are often held in German. Since human interpreters are too expensive for universities we are addressing this problem via speech translation technology deployed in KIT's lecture halls. Our simultaneous lecture translation system automatically translates lectures from German to English in real-time. Other supported language directions are English to Spanish, English to French, English to German and German to French. Automatic simultaneous translation is more than just the concatenation of automatic speech recognition and machine translation technology, as the input is an unsegmented, practically infinite stream of spontaneous speech. The lack of segmentation and the spontaneous nature of the speech makes it especially difficult to recognize and translate it with sufficient quality. In addition to quality, speed and latency are of the utmost importance in order for the system to enable students to follow lectures. In this paper we present our system that performs the task of simultaneous speech translation of university lectures by performing speech translation on a stream of audio in real-time and with low latency. The system features several techniques beyond the basic speech translation task, that make it fit for real-world use. Examples of these features are a continuous stream speech recognition without any prior segmentation of the input audio, punctuation prediction, run-on decoding and run-on translation with continuously updating displays in order to keep the latency as low as possible.

Lost in Interpreting: Speech Translation from Source or Interpreter?

“…German is a hybrid HMM-DNN model (DE ASR). The same system was used also by KIT Lecture Translator [21]. English is neural sequence-to-sequence ASR [22].…”

NMT-Based Segmentation and Punctuation Insertion for Real-Time Spoken Language Translation

“…Due to its low latency, this segmentation method has been used extensively in real-time applications such as Lecture Translator [3,12]. In this work, we compare the online module of our NMT-based segmentation and punctuation against the LM-based system.…”

“…We build an NMT-based segmentation and punctuation model for English, and integrate into a real-time spoken language translation system [12]. The performance of this model is analyzed in both offline and online scenarios.…”

Translation Quality Assessment of Online Translation Systems in Translating English to Thai on Phuket Tourism

“…Each language contains its culture of the language owner. By speaking only one language a loss of cultural diversity occurs (Müller et al, 2016). Thailand is an Asian country that has its own identity with many tourism characteristics including language, traditions and culture.…”