Online Localization of Multiple Moving Speakers in Reverberant Environments

Abstract

This paper addresses the problem of online multiple moving speakers localization in reverberant environments. The direct-path relative transfer function (DP-RTF), as defined by the ratio between the first taps of the convolutive transfer function (CTF) of two microphones, encodes the inter-channel direct-path information and is thus used as a localization feature being robust against reverberation. The CTF estimation is based on the cross-relation method. In this work, the recursive least-square method is proposed to solve the cross-relation problem, due to its relatively low computational cost and its good convergence rate. The DP-RTF feature estimated at each time-frequency bin is assumed to correspond to a single speaker. A complex Gaussian mixture model is used to assign each observed feature to one among several speakers. The recursive expectation-maximization algorithm is adopted to update online the model parameters. The method is evaluated with a new dataset containing multiple moving speakers, where the ground-truth speaker trajectories are recorded with a motion capture system.

Online Localization and Tracking of Multiple Moving Speakers in Reverberant Environments

“…• Since we deal with moving speakers or, more generally, with moving audio sources, DP-RTF features are computed using the online CTF estimation framework presented in [30], based on recursive least squares (RLS), rather than using the batch CTF estimation of [12] which assumes static audio sources. The online RLS algorithm has a faster convergence rate than the least mean squares (LMS) algorithms described in [8], [9].…”

“…This paper is an extended version of [30] which has proposed an online DP-RTF method that has been combined with REM to estimate the source directions. In this paper, while we keep the DP-RTF method of [30] we propose to use EG. The advantages of using EG instead of REM are described in detail in Section III.…”

“…In addition, even in a lownoise case, many TF bins are dominated by noise due to the sparsity of speech spectra. To classify the speech frames and noise frames, and to remove the noise, we use the inter-frame spectral subtraction algorithm proposed in [30], [33].…”

“…It is easy to see that in order to compute (30) and (31), two elements are needed: the predictive distribution (24) and the marginal filtering distribution at t − 1, p(s t−1 |o 1:t−1 ). Remarkably, as a consequence of the factorization (29), we can replace p(s t−1 |o 1:t−1 ) with q(s t−1 ) = N n=1 q(s t−1,n ) in (24) and compute the predictive distribution as follows:…”

Direction of Arrival Estimation for Reverberant Speech Based on Enhanced Decomposition of the Direct Sound

“…A different set of methods that overcome reverberation while facilitating a flexible array configuration, are based on relative transfer function estimation [7]- [9]. The direct-path component of the relative transfer-functions (DP-RTF) is typically estimated from the measured data.…”