Source/Filter Factorial Hidden Markov Model, With Application to Pitch and Formant Tracking

Abstract

Tracking vocal tract formant frequencies (f p ) and estimating the fundamental frequency (f 0 ) are two tracking problems that have been tackled in many speech processing works, often independently, with applications to articulatory parameters estimations, speech analysis/synthesis or linguistics. Many works assume an auto-regressive (AR) model to fit the spectral envelope, hence indirectly estimating the formant tracks from the AR parameters. However, directly estimating the formant frequencies, or equivalently the poles of the AR filter, allows to further model the smoothness of the desired tracks. In this paper, we propose a Factorial Hidden Markov Model combined with a vocal source/filter model, with parameters naturally encoding the f 0 and f p tracks. Two algorithms are proposed, with two different strategies: first, a simplification of the underlying model, with a parameter estimation based on variational methods, and second, a sparse decomposition of the signal, based on Non-negative Matrix Factorization methodology.The results are comparable to state-of-the-art formant tracking algorithms. With the use of a complete production model, the proposed systems provide robust formant tracks which can be used in various applications. The algorithms could also be extended to deal with multiple-speaker signals.

Time-varying quasi-closed-phase analysis for accurate formant tracking in speech signals

“…The estimation and tracking of VTRs from speech signals is a challenging problem that has many applications in various areas: in acoustic and phonetic analysis [1], [2], in voice morphing [3], in speech recognition [4], [5], in speech and singing voice synthesis [6], [7], in voice activity detection [8], and in designing hearing aids [9], [10]. Many algorithms of varying complexity have been proposed in the literature for tracking formants in speech signals [11]- [15]. A dynamic programming (DP)-based tracking algorithm with a heuristic cost function on the initial formant candidates estimated using conventional linear prediction (LP) analysis was used in [11], [12].…”

“…This two-stage approach has a detection stage, where an initial estimate of the VTRs is obtained, followed by a tracking stage. An integrated approach towards tracking was adopted in [13]- [15] using state-space methods such as Kalman filtering (KF) and the factorial hidden Markov model (FHMM). In both approaches, analysis of the signal for the accurate estimation (or modeling) of the vocal tract system is an important and necessary computational block.…”

“…In both approaches, analysis of the signal for the accurate estimation (or modeling) of the vocal tract system is an important and necessary computational block. However, it should be mentioned here that there are a few exceptions, such as [15], which uses a non-negative matrix factorization (NMF)based source-filter modeling of speech signals. Recently, deep learning-based techniques [16]- [18] have also been studied as alternatives to conventional statistical signal processing-based formant estimation and tracking methods.…”

Time-Varying Quasi-Closed-Phase Weighted Linear Prediction Analysis of Speech for Accurate Formant Detection and Tracking

“…Accurate tracking of formants in speech signals has potential applications in acoustic-phonetic analysis of speech signals, speech enhancement, formant-based speech synthesis, pronunciation correction [1][2][3][4][5]. Many algorithms of varying complexity have been proposed in the literature for tracking formants in speech signals [6][7][8][9][10]. A dynamic programming (DP) based tracking with a heuristic cost function on the initial formant candidates estimated using a conventional LP analysis is used in [6,7].…”

“…A dynamic programming (DP) based tracking with a heuristic cost function on the initial formant candidates estimated using a conventional LP analysis is used in [6,7]. An integrated approach towards tracking is adopted in [8][9][10] using state-space methods such as Kalman filtering (KF) and factorial hidden Markov model (FHMM). Most of these algorithms use an underlying linear prediction (LP) based modeling of speech signals, except in [10] which uses a nonnegative matrix factorization (NMF) based source-filter modeling of speech signals.…”