Maximum Penalized Likelihood Kernel Regression for Fast Adaptation

Abstract

Abstract-This paper proposes a nonlinear generalization of the popular maximum-likelihood linear regression (MLLR) adaptation algorithm using kernel methods. The proposed method, called maximum penalized likelihood kernel regression adaptation (MPLKR), applies kernel regression with appropriate regularization to determine the affine model transform in a kernel-induced high-dimensional feature space. Although this is not the first attempt of applying kernel methods to conventional linear adaptation algorithms, unlike most of other kernelized adaptation methods such as kernel eigenvoice or kernel eigen-MLLR, MPLKR has the advantage that it is a convex optimization and its solution is always guaranteed to be globally optimal. In fact, the adapted Gaussian means can be obtained analytically by simply solving a system of linear equations. From the Bayesian perspective, MPLKR can also be considered as the kernel version of maximum a posteriori linear regression (MAPLR) adaptation. Supervised and unsupervised speaker adaptation using MPLKR were evaluated on the Resource Management and Wall Street Journal 5K tasks, respectively, achieving a word error rate reduction of 23.6% and 15.5% respectively over the speaker-independently model.

Nonlinear connectivity by Granger causality

“…This requires the kernel model, employed by KGC, to be expressed in a maximum likelihood framework: it is a matter for further work. We refer the reader to Mak et al (2009) for a recent attempt to generalize in a nonlinear fashion the maximum-likelihood linear regression.…”

Acoustic Model Adaptation for Speech Recognition

“…Padmanabhan et al combined linear mapping and non-linear mapping to achieve robustness against the data insufficiency [68]. Mak et al applied kernel methods to estimate an affine mapping in a higher dimension space than that of features [55]. In this method, regularization techniques were used to achieve robustness against the data insufficiency.…”

Rapid feature space MLLR speaker adaptation with bilinear models

“…Many state of the art adaptation methods can help compensate for speaker variability, channel variability and content variability. Generally speaking, the model-based adaptation algorithm can be divided into three categories [1], speaker clustering based method which includes eigen-space based methods, Bayesian based method such as maximum a posteriori adaptation, and transformed based methods, such as maximum likelihood linear regression adaptation. These model-based methods need to change the speakerindependent HMM parameters, which can be computationally expensive and requires storing significant amount of data for the adapted speaker-dependent models.…”