Streaming Automatic Speech Recognition with Hybrid Architectures and Deep Neural Network Models

Abstract

Current state-of-the-art models based on Long Short-Term Memory (LSTM) networks have been extensively used in ASR to improve performance. However, using LSTMs under a streaming setup is not straightforward due to real-time constraints. In this paper we present a novel streaming decoder that includes a bidirectional LSTM acoustic model as well as an unidirectional LSTM language model to perform the decoding efficiently while keeping the performance comparable to that of an off-line setup. We perform a one-pass decoding using a sliding window scheme for a bidirectional LSTM acoustic model and an LSTM language model. This has been implemented and assessed under a pure streaming setup, and deployed into our production systems. We report WER and latency figures for the well-known Lib-riSpeech and TED-LIUM tasks, obtaining competitive WER results with low-latency responses.

Streaming Neural Speech Translation

“…Researchers of the MLLP ASR team collaborated on the papers that use the output of ASR systems instead of the reference transcriptions. Specifically, recent advances in streaming ASR technology (Jorge Cano 2022) allowed this…”