An Information Theoretic Combination of MFCC and TDOA Features for Speaker Diarization

Abstract

This correspondence describes a novel system for speaker diarization of meetings recordings based on the combination of acoustic features (MFCC) and Time Delay of Arrivals (TDOA). The first part of the paper analyzes differences between MFCC and TDOA features which possess completely different statistical properties. When Gaussian Mixture Models are used, experiments reveal that the diarization system is sensitive to the different recording scenarios (i.e. meeting rooms with varying number of microphones). In the second part, a new multistream diarization system is proposed extending previous work on Information Theoretic diarization. Both speaker clustering and speaker realignment steps are discussed; in contrary to current systems, the proposed method avoids to perform the feature combination averaging log-likelihood scores. Experiments on meetings data reveal that the proposed approach outperforms the GMM based system when the recording is done with varying number of microphones.

Speaker diarization of meetings based on large TDOA feature vectors

“…As previously pointed in [11], the IB system appears more robust to variation of weights across different meeting recordings. This robustness holds also in case of all-pairs TDOA feature vectors.…”

“…Whenever multiple features are available, the combination is performed in the space of relevance variables Y [11]. Separate GMMs with the same number of components are trained for each feature stream.…”

“…After clustering, the speaker boundaries are realigned. Instead of using HMM/GMMs, the realignment is performed in the space of relevance variables p(y|x) using a Kullback-Leibler divergence based HMM system described in [11].…”

Blind spatial sound source clustering and activity detection using uncalibrated microphone array

“…However, most of the existing methods assume that the microphone location is given to estimate the direction of arrival of speakers [3]- [6]. Some methods using Time Difference Of Arrival (TDOA) have been proposed [7]- [9], which do not assume the known microphone location. These methods propose using HMM for speaker segmentation and clustering, as well as hierarchical agglomerative clustering using spacial information.…”

“…These methods propose using HMM for speaker segmentation and clustering, as well as hierarchical agglomerative clustering using spacial information. However, the methods have difficulty with overlapping speech [7] and estimating the number of speakers deterministically [8], [9].…”