Investigating Sequence-Level Normalisation For CTC-Like End-to-End ASR

Abstract

End-to-end Automatic Speech Recognition (E2E ASR) significantly simplifies the training process of an ASR model. Connectionist Temporal Classification (CTC) is one of the most popular methods for E2E ASR training. Implicitly, CTC has a unique topology which is very useful for sequence modelling. However, we find that by changing to another topology, we can make it even more effective. In this paper, we propose a new CTC-like method, for E2E ASR training, by modifying the topology of original CTC, so that the wellknown abuse of the blank label in CTC can be resolved theoretically. As we change the topology, a normalisation term is necessary, which makes the form of the final loss function similar to Maximum Mutual Information (MMI); we hence name our method MMI-CTC. In addition to maximising the posterior probability of the target sequence, the normalisation enables models to explicitly minimise the probability of competing hypothesis at the word sequence level. Our experimental results show that MMI-CTC is more efficient than CTC, and that the normalisation is essential for sequence training.

Speech Recognition via CTC-CNN Model

“…(3) Combined with the CTC decoding algorithm, the end-to-end prediction of sequence data is effectively realized [10].…”

“…The recursive calculation formula of the backward probability obtained by recursion is shown in Eq. (10).…”

Regarding Topology and Variant Frame Rates for Differentiable WFST-based End-to-End ASR

“…Note that with most topologies (except CTC topology), not all the paths in E are valid, which makes the summation of the probabilities of all possible word sequences not equal to one. A previous work [22] has shown that the normalisation (the denominator term in eq. ( 2)) is crucial for the sequence-level loss function.…”

“…Note that in [26], the authors compared different CTC topologies, while in this work, we compare different topologies most of which are not equivalent to CTC. Inspired by previous work [22], we introduce an extra state for each phone to increase the modelling power, resulting in the S2-T1 topology in table 1, where there is no self-loop for the first state in S2-T1, and the second state is optional and skippable. S2-T1⋆ is similar to S2-T1, except for the self-loop on the first state.…”