Open-Set Short Utterance Forensic Speaker Verification Using Teacher-Student Network with Explicit Inductive Bias

Abstract

In forensic applications, it is very common that only small naturalistic datasets consisting of short utterances in complex or unknown acoustic environments are available. In this study, we propose a pipeline solution to improve speaker verification on a small actual forensic field dataset. By leveraging large-scale out-of-domain datasets, a knowledge distillation based objective function is proposed for teacher-student learning, which is applied for short utterance forensic speaker verification. The objective function collectively considers speaker classification loss, Kullback-Leibler divergence, and similarity of embeddings. In order to advance the trained deep speaker embedding network to be robust for a small target dataset, we introduce a novel strategy to fine-tune the pre-trained student model towards a forensic target domain by utilizing the model as a finetuning start point and a reference in regularization. The proposed approaches are evaluated on the 1 st 48-UTD forensic corpus, a newly established naturalistic dataset of actual homicide investigations consisting of short utterances recorded in uncontrolled conditions. We show that the proposed objective function can efficiently improve the performance of teacher-student learning on short utterances and that our fine-tuning strategy outperforms the commonly used weight decay method by providing an explicit inductive bias towards the pre-trained model.

Few-Shot Speaker Identification Using Lightweight Prototypical Network With Feature Grouping and Interaction

“…In this section, we compare the proposed lightweight method to six state-of-the-art methods for lightweight SV, including the ECAPA-TDNNLite [50], EfficientTDNN [51], KD-based [52], Thin-ResNet34 [64], Fast-ResNet34 [65], and CSTCTS1dConv (Channel Split Time-Channel-Time Separable 1-dimensional Convolution) [66]. The ECAPA-TDNNLite based method [50] is a lightweight version of the ECAPA-TDNN based method, in which a large model, ECAPA-TDNN, is utilized for enrollment and a small model, ECAPA-TDNNLite, is used for verification.…”

“…Additionally, some researchers applied the techniques of Knowledge Distillation (KD) [46], [47] and Neural Architecture Search (NAS) [48] to implement lightweight SV [49]- [52]. In the work of [49], the strategy of teacher-student training was proposed for text-independent SV, and competitive error rate with 88-93% smaller models was obtained.…”

A Supervised Learning Method for Improving the Generalization of Speaker Verification Systems by Learning Metrics from a Mean Teacher

“…They trained the model to decrease the distances between speaker embeddings extracted from the same speaker utterances and increase the distances between speaker embeddings of different speakers. In addition to these loss functions, the prior studies have investigated diverse methods, such as data augmentation [17,18], network architectures [19,20], and system frameworks [21,22].…”

DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning

“…For input-level, models usually can be adapted by training with enhanced [8] or domain-translated [9] input features. For adaptation at embedding-level, it often targets at minimizing certain distances between source and target domains to align them in the same embedding space, such as cosine distance [10], mean squared error (MSE) [11], and maximum mean discrepancy (MMD) [12]. However, this method usually requires parallel or artificial simulated data, which cannot generalize well to real-world scenarios.…”