Learning vocal tract variables with multi-task kernels

Abstract

The problem of acoustic-to-articulatory speech inversion continues to be a challenging research problem which significantly impacts automatic speech recognition robustness and accuracy. This paper presents a multi-task kernel based method aimed at learning Vocal Tract (VT) variables from the Mel-Frequency Cepstral Coefficients (MFCCs). Unlike usual speech inversion techniques based on individual estimation of each tract variable, the key idea here is to consider all the target variables simultaneously to take advantage of the relationships among them and then improve learning performance. The proposed method is evaluated using synthetic speech dataset and corresponding tract variables created by the TAsk Dynamics Application (TADA) model and compared to the hierarchical ε-SVR speech inversion technique.Index Terms-Multi-task learning, matrix-valued kernel, vocal tract variables, acoustic-to-articulatory inversion.

Operator-valued Kernels for Learning from Functional Response Data

“…The problem of speech inversion has received increasing attention in the speech processing community in the recent years (see Schroeter and Sondhi (1994); Mitra et al (2010); Kadri et al (2011a) and references therein). This problem, aka acoustic-articulatory inversion, involves inverting the forward process of speech production (see Figure 4).…”

“…In most cases, these works address the articulatory estimation problem within a single-task learning perspective. However, in Richmond (2007) and more recently in Kadri et al (2011a), the authors put forward the idea that we can benefit from viewing the acoustic-articulatory inversion problem from a multi-task learning perspective. Motivated by comparing our functional operator-valued kernel based approach with multivariate kernel methods, we report on experiments similar to those performed by Mitra et al (2009) and Kadri et al (2011a).…”

“…However, in Richmond (2007) and more recently in Kadri et al (2011a), the authors put forward the idea that we can benefit from viewing the acoustic-articulatory inversion problem from a multi-task learning perspective. Motivated by comparing our functional operator-valued kernel based approach with multivariate kernel methods, we report on experiments similar to those performed by Mitra et al (2009) and Kadri et al (2011a). The tract variables learning technique proposed by Mitra et al (2009) is based on a hierarchical ε-SVR architecture constructed by associating different SVRs, a SVR for each tract variable.…”

“…To consider the dependencies between VT time functions, the SVRs corresponding to independent VT variables are first created and then used for constructing the others. Otherwise, the acousticto-articulatory method in Kadri et al (2011a) is based on learning a vector-valued function using a matrix-valued kernel proposed in Caponnetto et al (2008).…”

Subject-independent acoustic-to-articulatory mapping of fricative sounds by using vocal tract length normalization

“…Although several attempts have been made during more than thirty years, the speech researchers still regard the acoustic-to-articulatory inversion as an open issue [2][3][4]. Roughly, inversion methods can be divided into two This paper seeks to show that using VTLN (Vocal Tract Length Normalization) in conjunction with statistically relevant parameters produces effective results for the case of acoustic-to-articulatory inversion in a speakerindependent way.…”