Spatial Pyramid Encoding with Convex Length Normalization for Text-Independent Speaker Verification

Abstract

In this paper, we propose a new pooling method called spatial pyramid encoding (SPE) to generate speaker embeddings for text-independent speaker verification. We first partition the output feature maps from a deep residual network (ResNet) into increasingly fine sub-regions and extract speaker embeddings from each sub-region through a learnable dictionary encoding layer. These embeddings are concatenated to obtain the final speaker representation. The SPE layer not only generates a fixed-dimensional speaker embedding for a variable-length speech segment, but also aggregates the information of feature distribution from multi-level temporal bins. Furthermore, we apply deep length normalization by augmenting the loss function with ring loss. By applying ring loss, the network gradually learns to normalize the speaker embeddings using model weights themselves while preserving convexity, leading to more robust speaker embeddings. Experiments on the VoxCeleb1 dataset show that the proposed system using the SPE layer and ring loss-based deep length normalization outperforms both ivector and d-vector baselines. Index Terms: speaker verification, spatial pyramid encoding, learnable dictionary encoding, ring loss, length normalization d-vector systemsWe can classify d-vector based SV systems according to the loss function used. The first one is based on the softmax loss defined in [23] as the combination of a cross-entropy loss, a softmax function and the last fully connected layer [7,8,24]. In this system, a speaker classifier is trained to classify speakers in the training set. The softmax loss encourages the separability of speaker embeddings. However, the softmax loss is not sufficient to learn the discriminative embedding with a large margin, and more researchers began to explore discriminative loss functions for enhanced generalization ability.

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

“…The TDNN is composed of 1D convolution and fully connected layers. Several later studies [16,19,21,22,25,27] suggest replacing the TDNN with variants of ResNet34, composed of 2D convolutions, as the encoder network. We used TDNN and ResNet34 for x-vector and LDE embeddings, respectively.…”

“…Villalba et al summarized several state-of-the-art speaker recognition systems for the NIST SRE18 Challenge [16], where x-vector based systems [17] consistently outperformed i-vector based systems [18]. There has also been a surge of interest in new encoding methods and endto-end loss functions for speaker recognition [19,20,21,22,23,24,25]. One prominent advancement is the use of learnable dictionary encoding (LDE) [19] and angular softmax [20] for speaker recognition, which are reported to boost the speaker recognition performance on open-source corpora such as the VoxCelebs [26,27].…”

Self-Adaptive Soft Voice Activity Detection Using Deep Neural Networks for Robust Speaker Verification

“…The first module is the frame-level feature extractor which takes a sequence of acoustic features x t and outputs corresponding speaker features h t (t = 1, · · · , T ). In our system, ResNet [19] is used as a feature extractor, which has been widely used in previous studies [20,21,22]. The architecture is described in Table 1.…”

“…The minibatch size is 64, and the weight decay parameter is 0.0001. We use the same learning rate schedule as in [22] with the initial learning rate of 0.1.…”

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

“…This assumes that voice activity detection (VAD) has not been applied to remove non-speech frames, which may degrade the SV performance, from the audio. Most SV studies still rely on a traditional energy-based VAD [13], [20], [30], and even some of them do not apply VAD [31], [32]. It is because most SV databases are included in the following two cases, thus minimizing the need of robust VAD: (1) They were recorded in relatively clean conditions, where the naive energy-based VAD performs reasonably well.…”