Improving Mandarin Speech Recogntion with Block-augmented Transformer

Abstract

Recently Convolution-augmented Transformer (Conformer)[1] has shown promising results in Automatic Speech Recognition (ASR), outperforming the previous best published Transformer Transducer[2]. In this work, we believe that the output information of each block in the encoder and decoder is not completely inclusive, in other words, their output information may be complementary. We study how to take advantage of the complementary information of each block in a parameter-efficient way, and it is expected that this may lead to more robust performance. Therefore we propose the Block-augmented Transformer for speech recognition, named Blockformer. We have implemented two block ensemble methods: the base Weighted Sum of the Blocks Output (Base-WSBO), and the Squeeze-and-Excitation module [3] to Weighted Sum of the Blocks Output (SE-WSBO). Experiments have proved that the Blockformer significantly outperforms the state-of-the-art Conformer-based models on AISHELL-1, our model achieves a CER of 4.35% without using a language model and 4.10% with an external language model on the testset.

Automatic speech recognition based on conformer with convolution enhanced channel attention block

“…Two methods of block integration were proposed by Xiaoming Ren et al [11] One approach involves weighted averaging, while the other employs SE modules for block integration. Their research demonstrates that the use of SE modules can significantly improve the accuracy of speech recognition models.…”

Automatic speech recognition based on attention-enhanced blockformer

“…Blockformer [11] improves upon Conformer by adopting the SE block presented in SENet [12] . Initially, it squeezes the output information of all the blocks through global average pooling to derive the representative information of each block, and then it activates the information of this value through the two nonlinear layers to make the value more effective in characterizing the corresponding block, and then the processed value is used as weights, multiplied by the output of the corresponding block, and finally the weighted output of each block is summed to obtain the final result.…”

New Unit Dot Product Similarity Method and Parallelized Greedy Soup Algorithm in The End-to-end Automatic Speech Recognition

“…Therefore, whether in streaming or non-stream speech recognition, the end-to-end speech recognition model based on Transformer has achieved very good results. [6,7] The self-attention mechanism in the Transformer model simulates human attention to important things. Various improvement approaches based on the Transformer model have become a hot topic in the research field.…”