Comparison of Decoding Strategies for CTC Acoustic Models

Abstract

Connectionist Temporal Classification has recently attracted a lot of interest as it offers an elegant approach to building acoustic models (AMs) for speech recognition. The CTC loss function maps an input sequence of observable feature vectors to an output sequence of symbols. Output symbols are conditionally independent of each other under CTC loss, so a language model (LM) can be incorporated conveniently during decoding, retaining the traditional separation of acoustic and linguistic components in ASR.For fixed vocabularies, Weighted Finite State Transducers provide a strong baseline for efficient integration of CTC AMs with n-gram LMs. Character-based neural LMs provide a straight forward solution for open vocabulary speech recognition and all-neural models, and can be decoded with beam search. Finally, sequence-to-sequence models can be used to translate a sequence of individual sounds into a word string.We compare the performance of these three approaches, and analyze their error patterns, which provides insightful guidance for future research and development in this important area.

Hybrid Autoregressive Transducer (HAT)

“…Except for the modeling unit, these models are very similar to conventional acoustic models and perform well when combined with an external LM during decoding (beam search). [23,24].…”

A Hardware-Oriented and Memory-Efficient Method for CTC Decoding

“…Some works use this decoding method to build the CTC-layers in their hardware architectures of RNNs [17]. Although this way can already provide useful transcriptions, its limited accuracy is not sufficient to meet the demands of many sequence tasks [26].…”

“…In ASR tasks, the traditional approach is based on HMMs [16], while recent works have shown great interest in building end-to-end models, using CTC-based deep RNNs. By training networks with large amounts of data, CTC-based models achieved great success [7], [11], [4], [12], [26], [18]. CTC is also widely used in other learning tasks such as handwriting recognition and scene text recognition, offering superior performance [8], [2], [19].…”

“…The RNN+CTC model is widely used, and the CTC beam search decoding algorithm is one of the most popular decoding methods [26]. However, the original beam search algorithm consumes a lot of memory space, making us believe that reducing storage consumption is very necessary.…”

Who Needs Words? Lexicon-Free Speech Recognition

“…The first lexicon-free beam-search decoder aiming at dealing with OOV was benchmarked on Switchboard [21], although with a significantly worse word error rate (WER) than lexicon-based systems. Other recent works in this direction include [22,23] on the English and [24, 25] on the Arabic and Finnish languages.Here, we study a simple end-to-end ASR system combining a character level acoustic model with a character level language model through beam search. We show that it can yield competitive word error rates on the WSJ and Librispeech corporas, even without a lexicon.…”