Differentiable Supervector Extraction for Encoding Speaker and Phrase Information in Text Dependent Speaker Verification

Abstract

In this paper, we propose a new differentiable neural network alignment mechanism for text-dependent speaker verification which uses alignment models to produce a supervector representation of an utterance. Unlike previous works with similar approaches, we do not extract the embedding of an utterance from the mean reduction of the temporal dimension. Our system replaces the mean by a phrase alignment model to keep the temporal structure of each phrase which is relevant in this application since the phonetic information is part of the identity in the verification task. Moreover, we can apply a convolutional neural network as front-end, and thanks to the alignment process being differentiable, we can train the whole network to produce a supervector for each utterance which will be discriminative with respect to the speaker and the phrase simultaneously. As we show, this choice has the advantage that the supervector encodes the phrase and speaker information providing good performance in text-dependent speaker verification tasks. In this work, the process of verification is performed using a basic similarity metric, due to simplicity, compared to other more elaborate models that are commonly used. The new model using alignment to produce supervectors was tested on the RSR2015-Part I database for text-dependent speaker verification, providing competitive results compared to similar size networks using the mean to extract embeddings. 1

Optimization of False Acceptance/Rejection Rates and Decision Threshold for End-to-End Text-Dependent Speaker Verification Systems

“…The corpus is divided into three speaker subset: background (bkg), development (dev), and evaluation (eval). Unlike our previous work [25,15], in this paper, we only employ the bkg data (97 speakers, 47 female/50 male) for training, and reserve the dev data to scores normalization. The evaluation part is used for enrollment and trial evaluation.…”

“…In the following section, we present the structure of the system used for experiments Fig.1. First, we describe the frontend based on neural networks combined with the differentiable alignment mechanism proposed in our previous work [25,15]. Then, the back-end strategies are described.…”

“…In the context of the text-dependent verification tasks [6,26], the averaging dismisses the order of phonetic information in the utterance. In [25,15], we addressed this problem by replacing the average pooling mechanism by a frame-to-state alignment method as a new layer into the DNN architecture. With this mechanism, the temporal structure of the uttered phrase Figure 1: Two architectures used to create the SV system.…”

Optimization of the area under the ROC curve using neural network supervectors for text-dependent speaker verification

“…However, this approach does not work efficiently in text-dependent tasks since the uttered phrase is a relevant piece of information to correctly determine the identities and the system has to detect a match in the speaker and the phrase to be correct [6,9]. In our previous work [10], we have noted that part of the imprecisions may be derived from the use of the average as a representation of the utterance, and how this problem can be solved by adding a new internal layer into the deep neural network architecture which uses an alignment method to encode the temporal structure of the phrase in a supervector. In this paper, we propose a generalization for the use of different alignment mechanisms that can be employed in combination with the deep neural network to generate a differentiable supervector with good performances as will be shown in the experiments.…”

“…In our previous work [10], we explored another reason for the lack of effectiveness in these tasks. The order of the phonetic information of the uttered phrase is relevant for the identification.…”