Gradient Boosting Learning of Hidden Markov Models

Abstract

In this paper, we present a new training algorithm, gradient boosting learning, for Gaussian mixture density (GMD) based acoustic models. This algorithm is based on a function approximation scheme from the perspective of optimization in function space rather than parameter space, i.e., stage-wise additive expansions of GMDs are used to search for optimal models instead of gradient descent optimization of model parameters. In the proposed approach, GMD starts from a single Gaussian and is built up by sequentially adding new components. Each new component is globally selected to produce optimal gain in the objective function. MLE and MMI are unified under the H-criterion, which is optimized by the extended BW (EBW) algorithm. A partial extended EM algorithm is developed for stage-wise optimization of new components. Experimental results on WSJ task demonstrate that the new algorithm leads to improved model quality and recognition performance.

Gradient boosting machines, a tutorial

“…When considering pattern recognition problem, one can efficiently assess tasks like speech and motion recognition with boosted temporal models like HMM (Hu et al, 2007; Du et al, 2011). Another important application is the extraction of relevant information from large amounts of data.…”

Significant of Gradient Boosting Algorithm in Data Management System

“…There are two bunches of promising neurorobotics applications for gradient boosting algorithms: the high-accuracy design acknowledgment applications and the ensemble-based neural reenactments. When considering pattern acknowledgment issues, one can proficiently survey errands like speaking and movement acknowledgment with boosted worldly models like Gee (Hu et al, 2007;Du et al, 2011). Another vital application is the extraction of significant data from great volumes of information.…”