Self-Supervised Speaker Verification Using Dynamic Loss-Gate and Label Correction

Abstract

For self-supervised speaker verification, the quality of pseudo labels decides the upper bound of the system due to the massive unreliable labels. In this work, we propose dynamic loss-gate and label correction (DLG-LC) to alleviate the performance degradation caused by unreliable estimated labels. In DLG, we adopt Gaussian Mixture Model (GMM) to dynamically model the loss distribution and use the estimated GMM to distinguish the reliable and unreliable labels automatically. Besides, to better utilize the unreliable data instead of dropping them directly, we correct the unreliable label with model predictions. Moreover, we apply the negative-pairs-free DINO framework in our experiments for further improvement. Compared to the best-known speaker verification system with self-supervised learning, our proposed DLG-LC converges faster and achieves 11.45%, 18.35% and 15.16% relative improvement on Vox-O, Vox-E and Vox-H trials of Voxceleb1 evaluation dataset.

Self-Supervised Open-Set Speaker Recognition with Laguerre–Voronoi Descriptors

“…Due to its popularity, it also has been implemented in open-set speaker recognition. Han et al [ 27 ] introduced a DINO-based framework that required two-stage training. The first stage includes training the DINO framework.…”

Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification

“…It means that loss-gate can effectively select reliable labels which are of benefit to the model. However, we also try to set different thresholds (1,3,5), and find that the choice of threshold also has a non-negligible impact on model performance [29]. Based on the estimated GMM, our proposed dynamic loss-gate (DLG) can adjust the threshold dynamically considering the current training situation and obtains better performance than LG which only adopts a fixed threshold during the whole training process.…”

“…S EAKER verification (SV) is a task that utilizes speech as the biometric feature to verify the speakers' identities. Recently, deep learning methods have been widely applied for speaker verification (SV) tasks and many efforts have been made such as various model architecture [2], [3], [4], [5], [6], training objection [7], [8], [9], pooling methods [10], [11] and Part of the results have been presented at Interspeech 2022 [1]. All the authors are with the X-Lance Lab, Department of Computer Science and Engineering & MoE Key Laboratory of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, 200240 P. R. China (e-mail:{hanbing97, zhengyang.chen, yanminqian}@sjtu.edu.cn) so on, to achieve excellent performance compared with traditional methods such as Gaussian Mixture Model-Universal Background Model (GMM-UBM) [12], i-vector [13].…”