End-to-End Speech Recognition: A Survey

Abstract

In the last decade of automatic speech recognition (ASR) research, the introduction of deep learning brought considerable reductions in word error rate of more than 50% relative, compared to modeling without deep learning. In the wake of this transition, a number of all-neural ASR architectures were introduced. These so-called end-to-end (E2E) models provide highly integrated, completely neural ASR models, which rely strongly on general machine learning knowledge, learn more consistently from data, while depending less on ASR domainspecific experience. The success and enthusiastic adoption of deep learning accompanied by more generic model architectures lead to E2E models now becoming the prominent ASR approach. The goal of this survey is to provide a taxonomy of E2E ASR models and corresponding improvements, and to discuss their properties and their relation to the classical hidden Markov model (HMM) based ASR architecture. All relevant aspects of E2E ASR are covered in this work: modeling, training, decoding, and external language model integration, accompanied by discussions of performance and deployment opportunities, as well as an outlook into potential future developments.

Investigating Methods to Improve Language Model Integration for Attention-Based Encoder-Decoder ASR Models

“…Here we show, how a seemingly very simple modification makes the AED model streamable and turns out to be very robust and competitive, specifically on long-form speech, in contrast to many other AED and transducer models [8][9][10][11][12]. Interestingly, the small modification leads to an equivalence to transducer models, and we study the exact modeling differences.…”

“…This is different to the standard transducer training, which performs a full sum over all alignment paths. The standard transducer training criterion cannot be applied easily here due to the alignment label dependencies [9,12].…”

“…INTRODUCTION & RELATED WORK Among the potential streaming models, there are the traditional HMM [1], CTC [2] and more recently transducer [3]. While many streamable attention-based encoder-decoder (AED) models were proposed [4][5][6][7][8], they are too complicated, relying on too much heuristics and not being robust enough in comparison to the transducer [9].…”

“…Similar chunking in the decoder has been done in [14][15][16][17][18][19] and similar chunking in the encoder has been done in [7,[20][21][22][23][24][25][26][27][28][29]. There are also other approaches to make self-attention in the encoder streamable [9,30,31].…”

Proceedings of the First Workshop on Bangla Language Processing (BLP-2023)

“…Modern end-to-end automatic speech recognition (E2E-ASR) systems have made remarkable strides, performing well across various types of data Gulati et al, 2020;Prabhavalkar et al, 2023). This success can be attributed to the advancement of deep learning techniques relying on different training strategies, highly dependent on large datasets.…”