Bayesian HMM Based x-Vector Clustering for Speaker Diarization

Díez, Mireia; Burget, Lukáš; Wang, Shuai; Rohdin, Johan; Černocký, Jaň

doi:10.21437/interspeech.2019-2813

Cited by 60 publications

(54 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An HMM is used to refine the clusters after an initial run of AHC. Previous approaches to using an HMM for diarisation have used the speaker embeddings as the observed variable [12,13]. This report proposes to incorporate the spatial location of the current active speaker into the HMM, by having the SSL vector as an additional observed variable.…”

Section: Hidden Markov Model Diarisationmentioning

confidence: 99%

See 1 more Smart Citation

Hidden Markov Model Diarisation with Speaker Location Information

Wong

Xiao

Gong

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speaker diarisation methods often rely on speaker embeddings to cluster together the segments of audio that are uttered by the same speaker. When the audio is captured using a microphone array, it is possible to estimate the locations of where the sounds originate from. This location information may be complementary to the speaker embeddings in the diarisation processes. This report proposes to extend the Hidden Markov Model (HMM) clustering method, to enable the use of speaker location information. The HMM observation log-likelihood for the speaker location can take the form of a KLdivergence, when the speaker location is represented as a discrete posterior distribution of the probabilities that the sound originated from each possible location. Experimental results on a Microsoft rich meeting transcription task show that using speaker location information with the proposed HMM modification can yield performance improvements over using speaker embeddings alone.

show abstract

Section: Hidden Markov Model Diarisationmentioning

confidence: 99%

“…Furthermore, each AHC merger iteration requires an optimisation of the HMM to be run, using the Expectation-Maximisation (EM) algorithm, which can be computationally expensive. Work in [13] separates out the HMM from the AHC. AHC is stopped when a measured similarity between the remaining clusters falls below a threshold.…”

Section: Introductionmentioning

confidence: 99%

Hidden Markov Model Diarisation with Speaker Location Information

Wong

Xiao

Gong

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Among all pyannote.audio alternatives, it is the most similar: written in Python, it provides most of the afore-This research was partly funded by the French National Research Agency (ANR) through the ODESSA (ANR-15-CE39-0010) and PLUM-COT (ANR-16-CE92-0025) projects. We would like to thank Claude Barras for providing the overlapped speech detection output corresponding to system L 1 in Table 2 of [1], Neville Ryant for the speaker diarization output of the winning submission to DIHARD 2019 [2,3], Marie Kunešová for the overlapped speech detection output corresponding to system "AMI test (all subsets) + dereverberation" in Table 2 of [4], and Sylvain Meignier for the speaker diarization output of [5] on ETAPE dataset. mentioned blocks, and goes all the way down to the actual evaluation of the system.…”

Section: Introductionmentioning

confidence: 99%

Pyannote.Audio: Neural Building Blocks for Speaker Diarization

Bredin¹,

Yin²,

Coria³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

193

128

View full text Add to dashboard Cite

We introduce pyannote.audio, an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it provides a set of trainable end-to-end neural building blocks that can be combined and jointly optimized to build speaker diarization pipelines. pyannote.audio also comes with pre-trained models covering a wide range of domains for voice activity detection, speaker change detection, overlapped speech detection, and speaker embedding -reaching state-of-the-art performance for most of them.

show abstract

“…In most of the approaches, this stage is still based on the Agglomerative Hierachical Clustering (AHC) [10] in combination with Probabilistic Linear Discriminat Analysis (PLDA) scoring [11]. Recently, spectral clustering [12] [13], and variatioanl bayesian clustering [14,15] have been introduced, showing promising result. Also, alternatives to the PLDA scoring have been introduced using neural networks that learn how to score two speech segments [16], using siamese networks [17] or Bi-LSTMs [18].…”

Section: Introductionmentioning

confidence: 99%

Supervised Online Diarization with Sample Mean Loss for Multi-Domain Data

Fini

Brutti

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Recently, a fully supervised speaker diarization approach was proposed (UIS-RNN) which models speakers using multiple instances of a parameter-sharing recurrent neural network. In this paper we propose qualitative modifications to the model that significantly improve the learning efficiency and the overall diarization performance. In particular, we introduce a novel loss function, we called Sample Mean Loss and we present a better modelling of the speaker turn behaviour, by devising an analytical expression to compute the probability of a new speaker joining the conversation. In addition, we demonstrate that our model can be trained on fixed-length speech segments, removing the need for speaker change information in inference. Using x-vectors as input features, we evaluate our proposed approach on the multi-domain dataset employed in the DIHARD II challenge: our online method improves with respect to the original UIS-RNN and achieves similar performance to an offline agglomerative clustering baseline using PLDA scoring.

show abstract

Bayesian HMM Based x-Vector Clustering for Speaker Diarization

Cited by 60 publications

References 10 publications

Hidden Markov Model Diarisation with Speaker Location Information

Hidden Markov Model Diarisation with Speaker Location Information

Pyannote.Audio: Neural Building Blocks for Speaker Diarization

Supervised Online Diarization with Sample Mean Loss for Multi-Domain Data

Contact Info

Product

Resources

About