Adaptive kernel canonical correlation analysis for estimation of task dynamics from acoustics

Rudzicz, Frank

doi:10.1109/icassp.2010.5495697

Cited by 9 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A common approach to extracting this space is through canonical correlation analysis (CCA), which finds pairs of maximally correlated projections of the data in the two views. CCA has been successfully applied to various tasks in speech [21,22,23,24,25,26], natural language processing [27], and computer vision [28,29]. A closely related technique is Partial Least Squares (PLS), which finds pairs of maximally covarying projections of the data in the two views.…”

Section: Stochastic Optimization For Plsmentioning

confidence: 99%

Stochastic optimization for PCA and PLS

Arora

Cotter

Livescu

et al. 2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

118

144

View full text Add to dashboard Cite

show abstract

Section: Stochastic Optimization For Plsmentioning

confidence: 99%

Stochastic optimization for PCA and PLS

Arora

Cotter

Livescu

et al. 2012

2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

118

144

View full text Add to dashboard Cite

show abstract

“…In this work, we applied one exemplar-based speaker independent acoustic-to-articulatory inversion methods based on Ghosh and Narayanan (2011) and one deep neural network (DNN) based approach based on Uria et al (2011) to generate the estimated articulatory signals. It is worth noting that other types of acoustic-to-articulatory mapping, such as CCA (Bharadwaj et al, 2012; Arora and Livescu, 2013), Kernel CCA (Rudzicz, 2010; Arora and Livescu, 2013), Gaussian Mixture Model (GMM) (Ghosh and Narayanan, 2013; Ozbek et al, 2011; Özbek et al, 2012), attributes classification (Leung et al, 2004; Zhang et al, 2007; Siniscalchi et al, 2013, 2012) and articulatory phonological code (Zhuang et al, 2009), etc., could also be applied here. The reason to choose the exemplar-based speaker independent acoustic-to-articulatory inversion methods is that we can directly compare the performance against the real articulatory trajectories measurement to find out the gap which shows the potential for better speaker aware inversion techniques.…”

Section: Introductionmentioning

confidence: 99%

Speaker verification based on the fusion of speech acoustics and inverted articulatory signals

Kim

Lammert

et al. 2016

Computer Speech & Language

View full text Add to dashboard Cite

We propose a practical, feature-level and score-level fusion approach by combining acoustic and estimated articulatory information for both text independent and text dependent speaker verification. From a practical point of view, we study how to improve speaker verification performance by combining dynamic articulatory information with the conventional acoustic features. On text independent speaker verification, we find that concatenating articulatory features obtained from measured speech production data with conventional Mel-frequency cepstral coefficients (MFCCs) improves the performance dramatically. However, since directly measuring articulatory data is not feasible in many real world applications, we also experiment with estimated articulatory features obtained through acoustic-to-articulatory inversion. We explore both feature level and score level fusion methods and find that the overall system performance is significantly enhanced even with estimated articulatory features. Such a performance boost could be due to the inter-speaker variation information embedded in the estimated articulatory features. Since the dynamics of articulation contain important information, we included inverted articulatory trajectories in text dependent speaker verification. We demonstrate that the articulatory constraints introduced by inverted articulatory features help to reject wrong password trials and improve the performance after score level fusion. We evaluate the proposed methods on the X-ray Microbeam database and the RSR 2015 database, respectively, for the aforementioned two tasks. Experimental results show that we achieve more than 15% relative equal error rate reduction for both speaker verification tasks.

show abstract

“…The intuition is that articulatory measurements provide information about the linguistic content, and that much of the non-discriminative information in the two views is largely uncorrelated and therefore filtered out. CCA/KCCA have also been used with audio and video for speaker clustering [22] and identification [23]; for speaker normalization [24], where the views are the speakers; for articulatory inversion [25]; and to study critical articulators [26].…”

Section: Introductionmentioning

confidence: 99%

Multi-view CCA-based acoustic features for phonetic recognition across speakers and domains

Arora

Livescu

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

Canonical correlation analysis (CCA) and kernel CCA can be used for unsupervised learning of acoustic features when a second view (e.g., articulatory measurements) is available for some training data, and such projections have been used to improve phonetic frame classification. Here we study the behavior of CCA-based acoustic features on the task of phonetic recognition, and investigate to what extent they are speakerindependent or domain-independent. The acoustic features are learned using data drawn from the University of Wisconsin X-ray Microbeam Database (XRMB). The features are evaluated within and across speakers on XRMB data, as well as on out-of-domain TIMIT and MOCHA-TIMIT data. Experimental results show consistent improvement with the learned acoustic features over baseline MFCCs and PCA projections. In both speaker-dependent and cross-speaker experiments, phonetic error rates are improved by 4-9% absolute (10-23% relative) using CCA-based features over baseline MFCCs. In cross-domain phonetic recognition (training on XRMB and testing on MOCHA or TIMIT), the learned projections provide smaller improvements.

show abstract

Adaptive kernel canonical correlation analysis for estimation of task dynamics from acoustics

Cited by 9 publications

References 13 publications

Stochastic optimization for PCA and PLS

Stochastic optimization for PCA and PLS

Speaker verification based on the fusion of speech acoustics and inverted articulatory signals

Multi-view CCA-based acoustic features for phonetic recognition across speakers and domains

Contact Info

Product

Resources

About