Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors

Abstract

A method to perform offline and online speaker diarization for an unlimited number of speakers is described in this paper. End-to-end neural diarization (EEND) has achieved overlap-aware speaker diarization by formulating it as a multilabel classification problem. It has also been extended for a flexible number of speakers by introducing speaker-wise attractors. However, the output number of speakers of attractorbased EEND is empirically capped; it cannot deal with cases where the number of speakers appearing during inference is higher than that during training because its speaker counting is trained in a fully supervised manner. Our method, EEND-GLA, solves this problem by introducing unsupervised clustering into attractor-based EEND. In the method, the input audio is first divided into short blocks, then attractor-based diarization is performed for each block, and finally, the results of each block are clustered on the basis of the similarity between locallycalculated attractors. While the number of output speakers is limited within each block, the total number of speakers estimated for the entire input can be higher than the limitation. To use EEND-GLA in an online manner, our method also extends the speaker-tracing buffer, which was originally proposed to enable online inference of conventional EEND. We introduce a blockwise buffer update to make the speaker-tracing buffer compatible with EEND-GLA. Finally, to improve online diarization, our method improves the buffer update method and revisits the variable chunk-size training of EEND. The experimental results demonstrate that EEND-GLA can perform speaker diarization of an unseen number of speakers in both offline and online inferences.

Online Target Speaker Voice Activity Detection for Speaker Diarization

“…To make the number of output speakers flexible and unlimited, EEND-vector clustering (EEND-VC) [12]- [14] integrates end-to-end and clustering approaches, which deploys an EEND model for shortly divided audio blocks and then matches the inter-block speaker labels by clustering on speaker embeddings. In addition, EEND-GLA [15], [16] calculates local attractors from each short block and finds the speaker correspondence based on similarities between inter-block attractors. As its training only requires relative speaker labels within the recording, EEND-GLA is practical for adapting models on in-the-wild datasets without globally unique speaker labels.…”

“…Nevertheless, permutationinvariant training in EEND-based methods causes performance degradation when the number of speakers increases in long audios. Although a few studies [12]- [16] have explored the unsupervised clustering to address this problem, their results are still unsatisfactory. Recently, Target-Speaker Voice Activity Detection (TS-VAD) approaches [17]- [20] become attractive, which combines advantage of modularized methods and endto-end neural networks.…”

Robust End-to-end Speaker Diarization with Generic Neural Clustering

“…Another approach to tackle the issue of speaker permutation ambiguity involves maintaining acoustic features within a speaker-tracing buffer [26,30]. On top of this approach, Horiguchi et al [31] enhanced EEND-GLA for online usage by integrating a speaker-tracing buffer. Remarkably, this method achieved SOTA performance across various datasets, outperforming numerous offline modularized speaker diarization systems.…”

“…During training stage, this inconsistency can be solved using a permutation free objective [20,29], but it limits the EEND to be extended for online purpose as the order of speakers are always changing as a new signal coming. One solution is to employ a buffer to store both prior inputs and results to align the current results with previous one [26,30,31], but this usually requires a large buffer for a satisfying performance.…”

Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractor

“…Additionally, Horiguchi et al [21] introduced the EEND-EDA system, which utilizes an LSTM encoder-decoder network to model attractors for each speaker. Furthermore, researchers also proposed two-stage hybrid systems [22], [23] to address the challenge of handling a flexible number of speakers. These systems first output diarization results for short segments with a limited number of speakers using EEND, and then employ a clustering algorithm to solve the inter-segment speaker permutation problem.…”