Speech translation enhanced automatic speech recognition

Abstract

Nowadays official documents have to be made available in many languages, like for example in the EU with its 20 official languages. Therefore, the need for effective tools to aid the multitude of human translators in their work becomes easily apparent. An ASR system, enabling the human translator to speak his translation in an unrestricted manner, instead of typing it, constitutes such a tool. In this work we improve the recognition performance of such an ASR system on the target language of the human translator by taking advantage of an either written or spoken source language representation. To do so, machine translation techniques are used to translate between the different languages and then the involved ASR systems are biased towards the gained knowledge. We present an iterative approach for ASR improvement and outperform our baseline system by a relative word error rate reduction of 35.8% / 29.9% in the case of a written / spoken source language representation. Further, we show how multiple target languages, as for example provided by different simultaneous translators during European Parliament debates, can be incorporated into our system design for an improvement of all involved ASR systems.

Extracting clues from human interpreter speech for spoken language translation

“…We report promising results for initial experiments conducted on a small English↔Spanish EPPS development set. In previous work [1], we showed dramatic improvements in ASR and SLT when incorporating information from human interpretations. These interpretations were artificially created by collecting sentences read from the bilingual Basic Travel Expression Corpus (BTEC).…”

“…This architecture allows for a mutual adaptation of all involved SLT components. It is based on the system architecture described in [1]. In step 1, we automatically transcribe the English and Spanish speech.…”

Speech Translation Enhanced ASR for European Parliament Speeches - On the Influence of ASR Performance on Speech Translation

“…The goal of speech translation enhanced ASR (STE-ASR) as introduced in [1] and [14] is to improve the speech recognition performance in one language, regardless of whether the speech comes from the original speaker or an interpreter, by making use of all available parallel speech and other information (e.g. in the form of documents) in all available languages.…”

Recovery of acronyms, out-of-lattice words and pronunciations from parallel multilingual speech

“…Despite this, several authors have tried to combine ASR and MT in a parallel fashion. Some of these methods are used to combine speech with a text stream, usually for an application such as machine-aided human translation [2,3], although a few works have considered combining multiple speech streams [4,5].…”