Improving Mandarin Tone Recognition Using Convolutional Bidirectional Long Short-Term Memory with Attention

Abstract

Automatic tone recognition is useful for Mandarin spoken language processing. However, the complex F0 variations from the tone co-articulations and the interplay effects among tonality make it rather difficult to perform tone recognition of Chinese continuous speech. This paper explored the application of Bidirectional Long Short-Term Memory (BLSTM), which had the capability of modeling time series, to Mandarin tone recognition to handle the tone variations in continuous speech. In addition, we introduced attention mechanism to guide the model to select the suitable context information. The experimental results showed that the performance of proposed CNN-BLSTM with attention mechanism was the best and it achieved the tone error rate (TER) of 9.30% with a 17.6% relative error reduction from the DNN baseline system with TER of 11.28%. It demonstrated that our proposed model was more effective to handle the complex F0 variations than other models.

End-to-End Mandarin Tone Classification with Short Term Context Information

“…Frame-based approach feeds a sequence frames directly to the classifier, which outputs a sequence of labels. To capture context information, RNN [3,13] and CNN [3] are frequently used. Also, frame-based frameworks often use techniques such as pooling [12], attention [3], or Connectionist Temporal Classification (CTC) [13] to perform frame-level alignment to correctly output a series of tone labels.…”

“…To capture context information, RNN [3,13] and CNN [3] are frequently used. Also, frame-based frameworks often use techniques such as pooling [12], attention [3], or Connectionist Temporal Classification (CTC) [13] to perform frame-level alignment to correctly output a series of tone labels. This approach allows a single training pass to cover both the alignment and classification task, and does not require a pretrained ASR model.…”

“…T1 to T4 refer to the lexical tones. T1 is a flat tone with a relatively high frequency, T2 has a rising frequency, T3 first dips and then rises, and T4's frequency is falling [1,2,3,4,5]. In addition to these four tones, there's also a neutral tone, which itself doesn't possess a particular pitch contour pattern.…”

“…Thus, the pitch contours of a syllable can be affected by the context bi-directionally [5]. This phenomenon is called tonal coarticulation [3,5,6,8,9].…”

Standard Yorùbá context dependent tone identification using Multi-Class Support Vector Machine (MSVM)

“…Consequently, the design and implementation of Multi-class Support Vector Machine in the recognition of SY context dependent tone is presented in this paper to engender and provide arguments for the use of context dependent tone segment for SY ASR. In language such as SY, tones are associated with syllable (Yang and Zhang, 2018). SY has seven possible syllable structures, these include consonant -vowel , , digraph-vowel nasal , digraph-vowel , vowel , vowel nasal and syllabic nasal .…”