Unsupervised Methods for Speaker Diarization: An Integrated and Iterative Approach

Abstract

Abstract-In speaker diarization, standard approaches typically perform speaker clustering on some initial segmentation before refining the segment boundaries in a re-segmentation step to obtain a final diarization hypothesis. In this paper, we integrate an improved clustering method with an existing re-segmentation algorithm and, in iterative fashion, optimize both speaker cluster assignments and segmentation boundaries jointly. For clustering, we extend our previous research using factor analysis for speaker modeling. In continuing to take advantage of the effectiveness of factor analysis as a front-end for extracting speaker-specific features (i.e., i-vectors), we develop a probabilistic approach to speaker clustering by applying a Bayesian Gaussian Mixture Model (GMM) to principal component analysis (PCA)-processed i-vectors. We then utilize information at different temporal resolutions to arrive at an iterative optimization scheme that, in alternating between clustering and re-segmentation steps, demonstrates the ability to improve both speaker cluster assignments and segmentation boundaries in an unsupervised manner. Our proposed methods attain results that are comparable to those of a state-of-the-art benchmark set on the multi-speaker CallHome telephone corpus. We further compare our system with a Bayesian nonparametric approach to diarization and attempt to reconcile their differences in both methodology and performance.

Investigation of Segmentation in i-Vector Based Speaker Diarization of Telephone Speech

“…The most common approach consists of the segmentation of the input signal, followed by the merging of the segments into clusters corresponding to the individual speakers [1,2]. The alternative is to combine the segmentation and clustering steps into a single iterative process [3,4].…”

Bayesian analysis of similarity matrices for speaker diarization

“…The idea of using strong prior information obtained from large datasets was further developed in studies that first adopted i-vectors as features for speaker diarization [6,7,8]. For example, [6] not only showed that diarization can be successfully carried out directly in the i-vector space, but it also achieves state-of-the-art performance on conversational telephone data from the 2008 NIST SRE corpus.…”

“…For example, [6] not only showed that diarization can be successfully carried out directly in the i-vector space, but it also achieves state-of-the-art performance on conversational telephone data from the 2008 NIST SRE corpus. Unlike [4], model selection in [8] was performed in terms of iterative variational lower bound maximization, defined for the full recording.…”

“…In contrast to [8], we do not carry out diarization in the i-vector space directly, but use a similarity matrix constructed from pairwise segment scores in the given recording (Fig. 1).…”

“…In [8], Gaussian mixture model (GMM) based clustering was applied to speaker diarization, treating i-vectors as points in Eucludean space regardless of their nature. Compared to [8], the main advantage of the proposed approach is that the model uses additional prior knowledge from learned speaker verification metric.…”