Deep multi-metric learning for text-independent speaker verification

Abstract

Text-independent speaker verification is an important artificial intelligence problem that has a wide spectrum of applications, such as criminal investigation, payment certification, and interest-based customer services. The purpose of text-independent speaker verification is to determine whether two given uncontrolled utterances originate from the same speaker or not. Extracting speech features for each speaker using deep neural networks is a promising direction to explore and a straightforward solution is to train the discriminative feature extraction network by using a metric learning loss function. However, a single loss function often has certain limitations. Thus, we use deep multi-metric learning to address the problem and introduce three different losses for this problem, i.e., triplet loss, n-pair loss and angular loss. The three loss functions work in a cooperative way to train a feature extraction network equipped with Residual connections and squeeze-and-excitation attention. We conduct experiments on the large-scale VoxCeleb2 dataset, which contains over a million utterances from over 6, 000 speakers, and the proposed deep neural network obtains an equal error rate of 3.48%, which is a very competitive result. Codes for both training and testing and pretrained models are available at https://github.com/GreatJiweix/DmmlTiSV, which is the first publicly available code repository for large-scale text-independent speaker verification with performance on par with the state-of-the-art systems.

Improving robustness of one-shot voice conversion with deep discriminative speaker encoder

“…A ResNet-SE block mainly consists of convolution layers. Filters in the convolution layer explicitly model local features and allow spatial translation invariance, which make convolution layer suitable to extract frame level features [28]. SE block expands the temporal context of the frame level information by modeling channel interdependence in features, which has been verified to be helpful in speaker verification task [28].…”

“…In this paper, to further improve the effectiveness of speaker embedding extracted from only one utterance of an unseen speaker, we propose a deep discriminative speaker encoder. Inspired by [28], first residual network and squeezeand-excitation network [29] are integrated to extract discriminative frame level speaker information by modeling frame-wise and channel-wise interdependence in features. Then attention mechanism is introduced to give different weights to frame level speaker information.…”

TMS: A Temporal Multi-scale Backbone Design for Speaker Embedding

“…S PEAKER embedding is widely used as a front-end processing for speaker discriminative information extraction for speech application systems where speaker information is needed, for example, speaker verification systems in authentication for security access [1], speaker diarization systems in real-time meeting recordings and/or dialogs [2], [3]. Due to the success of deep learning frameworks in speech and image processing, speaker embedding algorithms have been proposed in which outputs of bottleneck layers could be used as speaker representation.…”

RSKNet-MTSP: Effective and Portable Deep Architecture for Speaker Verification

“…Timedelay neural network (TDNN) [5,6,7,8,9] and one-dimensional convolutional neural network (CNN) along the time axis, are representative frame-level structures. The segment-level structure [10,11,12,13,14,15] consider the input acoustic features as a grayscale image which has three dimensions for time, frequency and channel, respectively, and employ two-dimensional CNN to produce three-dimensional outputs. In segment-level structure, with the downsampling operation, the dimension number of time and frequency dimensions will decrease along with the increase of channel dimensions.…”