A pitch extraction algorithm tuned for automatic speech recognition

Abstract

In this paper we present an algorithm that produces pitch and probability-of-voicing estimates for use as features in automatic speech recognition systems. These features give large performance improvements on tonal languages for ASR systems, and even substantial improvements for non-tonal languages. Our method, which we are calling the Kaldi pitch tracker (because we are adding it to the Kaldi ASR toolkit), is a highly modified version of the getf0 (RAPT) algorithm. Unlike the original getf0 we do not make a hard decision whether any given frame is voiced or unvoiced; instead, we assign a pitch even to unvoiced frames while constraining the pitch trajectory to be continuous. Our algorithm also produces a quantity that can be used as a probability of voicing measure; it is based on the normalized autocorrelation measure that our pitch extractor uses. We present results on data from various languages in the BABEL project, and show a large improvement over systems without tonal features and systems where pitch and POV information was obtained from SAcC or getf0.

Atom decomposition-based intonation modelling

“…Rillard [25] and d'Allessandro [26] have suggested using the power of the speech signal instead, easing wRMSE calculation. We have opted for the latter, augmenting it with the POV calculated as detailed by Ghahremani [27]. Incorporating the POV in the weighing eliminates the need to hard threshold the POV to obtain voicing, making the whole approach more robust.…”

“…The Kaldi pitch tracker was used for F0 and probability of voicing (POV) extraction [27]. We used 50ms frame length with 5ms frameshift for extraction.…”

Intonation modelling using a muscle model and perceptually weighted matching pursuit

“…We will use the WCORR norm , in order to assess the perceptual quality of the modelled F 0 using the thresholds discussed in Section 4.2. To extract the continuous F 0 and POV estimates we will use the pitch tracker implemented in Kaldi (Ghahremani et al, 2014) 3 . The second hypothesis is one of comparison of our generalised CR model with a state-of-the-art implementation of the standard CR model.…”

“…3. We define the weighting function to be (4), where p(i) is the probability of voicing (POV), as defined by Ghahremani et al (2014), and e(i) is the energy contour of the speech signal. This is in accord with newer trends in perceptual intonation studies d'Alessandro et al, 2011).…”

ASR Systems as Models of Phonetic Category Perception in Adults

“…The same Kaldi recipe was used (see https://github.com/ bootphon/abkhazia/blob/master/abkhazia/kaldi/ kaldi templates/train and decode.sh) with the same parameters and input features to train all models. Input features consisted of 13 MFCC coefficients plus 3 pitchrelated features (Ghahremani et al, 2014) and their delta and delta-deltas coefficients. Pitch features were included because tone is contrastive in Mandarin and Vietnamese (i.e.…”