Pre-training of Speaker Embeddings for Low-latency Speaker Change Detection in Broadcast News

Sarı, Leda; Thomas, Samuel; Hasegawa‐Johnson, Mark; Picheny, Michael

doi:10.1109/icassp.2019.8683612

Cited by 11 publications

(22 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, in [41], probabilistic linear discriminant analysis (PLDA) based clustering is used for speaker diarization. In [30], i-vectors of two consecutive speech segments are concatenated and fed into a same/different classifier that determines if they are spoken by the same speaker. If the system output is 'different,' then it is declared that the beginning time of the second window is a change point.…”

Section: B Speaker Change Detectionmentioning

confidence: 99%

“…The proposed system for joint speaker adaptation and change detection combines the speaker adaptation scheme of [34] with Siamese change detection [30], and introduces an attention block to the auxiliary network. The attention block allows us to integrate ideas from the Siamese network into the new joint model.…”

Section: Joint Speaker Adaptation and Change Detectionmentioning

confidence: 99%

“…The Siamese network architecture used in our previous work [30] is based on the idea of triplet loss training of [28] followed by a same/different classifier. Fig.…”

Section: B Speaker Change Detection With Siamese Networkmentioning

confidence: 99%

“…For more information, see https://creativecommons.org/licenses/by/4.0/ approaches that fit a model to the features of individual segments and their concatenation, and choose the hypothesis with a higher score; this score can be the Bayesian information criterion (BIC) [26] or the Gaussian likelihood score [27]. Although i-vectors have been predominantly used to capture speaker information within segments, DNN-based speaker embeddings such as Siamese embeddings [28]- [30] and x-vectors [31]- [33] have become popular recently.…”

Section: Introductionmentioning

confidence: 99%

“…In this work, the problem we address is whether we can construct a single acoustic model that can detect the speaker change and automatically adapt to different speakers simultaneously. In order to develop such a system, we combine ideas from neural network based speaker change detection [30] and unsupervised speaker adaptation by an auxiliary network [12], [34]. The key novel contribution of this work is a method that moves the speaker-change decision inside the speech recognition system, in the form of a soft-decision speaker-attention layer.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Auxiliary Networks for Joint Speaker Adaptation and Speaker Change Detection

Sarı

Hasegawa‐Johnson

Thomas

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

Speaker adaptation and speaker change detection have both been studied extensively to improve automatic speech recognition (ASR). In many cases, these two problems are investigated separately: speaker change detection is implemented first to obtain single-speaker regions, and speaker adaptation is then performed using the derived speaker segments for improved ASR. However, in an online setting, we want to achieve both goals in a single pass. In this study, we propose a neural network architecture that learns a speaker embedding from which it can perform both speaker adaptation for ASR and speaker change detection. The proposed speaker embedding is computed using self-attention based on an auxiliary network attached to a main ASR network. ASR adaptation is then performed by subtracting, from the main network activations, a segment dependent affine transformation of the learned speaker embedding. In experiments on a broadcast news dataset and the Switchboard conversational dataset, we test our system on utterances with a change point in them and show that the proposed method achieves significantly better performance as compared to the unadapted main network (10-14% relative reduction in word error rate (WER)). The proposed architecture also outperforms three different speaker segmentation methods followed by ASR (around 10% relative reduction in WER).

show abstract

Section: B Speaker Change Detectionmentioning

confidence: 99%

Section: Joint Speaker Adaptation and Change Detectionmentioning

confidence: 99%

“…The Siamese network architecture used in our previous work [30] is based on the idea of triplet loss training of [28] followed by a same/different classifier. Fig.…”

Section: B Speaker Change Detection With Siamese Networkmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Auxiliary Networks for Joint Speaker Adaptation and Speaker Change Detection

Sarı

Hasegawa‐Johnson

Thomas

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

Self Cite

View full text Add to dashboard Cite

show abstract

Latent Space Representation for Multi-Target Speaker Detection and Identification with a Sparse Dataset Using Triplet Neural Networks

Cheuk

Balamurali

Roig

et al. 2019

2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

View full text Add to dashboard Cite

We present an approach to tackle the speaker recognition problem using Triplet Neural Networks. Currently, the ivector representation with probabilistic linear discriminant analysis (PLDA) is the most commonly used technique to solve this problem, due to high classification accuracy with a relatively short computation time. In this paper, we explore a neural network approach, namely Triplet Neural Networks (TNNs), to built a latent space for different classifiers to solve the Multi-Target Speaker Detection and Identification Challenge Evaluation 2018 (MCE 2018) dataset. This training set contains i-vectors from 3,631 speakers, with only 3 samples for each speaker, thus making speaker recognition a challenging task. When using the train and development set for training both the TNN and baseline model (i.e., similarity evaluation directly on the i-vector representation), our proposed model outperforms the baseline by 23%. When reducing the training data to only using the train set, our method results in 309 confusions for the Multi-target speaker identification task, which is 46% better than the baseline model. These results show that the representational power of TNNs is especially evident when training on small datasets with few instances available per class.

show abstract

Multichannel Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking Of Acoustic And Spatial Features

Hogg

Evers

Naylor

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

An essential part of any diarization system is the task of speaker segmentation which is important for many applications including speaker indexing and automatic speech recognition (ASR) in multi-speaker environments. Segmentation of overlapping speech has recently been a key focus of this work. In this paper we explore the use of a new multimodal approach for overlapping speaker segmentation that tracks both the fundamental frequency (F0) of the speaker and the speaker's direction of arrival (DOA) simultaneously. Our proposed multiple hypothesis tracking system, which simultaneously tracks both features, shows an improvement in segmentation performance when compared to tracking these features separately. An illustrative example of overlapping speech demonstrates the effectiveness of our proposed system. We also undertake a statistical analysis on 12 meetings from the AMI corpus and show an improvement in the HIT rate of 14.1% on average against a commonly used deep learning bidirectional long short term memory network (BLSTM) approach.

show abstract

Pre-training of Speaker Embeddings for Low-latency Speaker Change Detection in Broadcast News

Cited by 11 publications

References 14 publications

Auxiliary Networks for Joint Speaker Adaptation and Speaker Change Detection

Auxiliary Networks for Joint Speaker Adaptation and Speaker Change Detection

Latent Space Representation for Multi-Target Speaker Detection and Identification with a Sparse Dataset Using Triplet Neural Networks

Multichannel Overlapping Speaker Segmentation Using Multiple Hypothesis Tracking Of Acoustic And Spatial Features

Contact Info

Product

Resources

About