Self-Adaptive Soft Voice Activity Detection Using Deep Neural Networks for Robust Speaker Verification

Abstract

Voice activity detection (VAD), which classifies frames as speech or non-speech, is an important module in many speech applications including speaker verification. In this paper, we propose a novel method, called self-adaptive soft VAD, to incorporate a deep neural network (DNN)-based VAD into a deep speaker embedding system. The proposed method is a combination of the following two approaches. The first approach is soft VAD, which performs a soft selection of frame-level features extracted from a speaker feature extractor. The frame-level features are weighted by their corresponding speech posteriors estimated from the DNN-based VAD, and then aggregated to generate a speaker embedding. The second approach is self-adaptive VAD, which fine-tunes the pre-trained VAD on the speaker verification data to reduce the domain mismatch. Here, we introduce two unsupervised domain adaptation (DA) schemes, namely speech posteriorbased DA (SP-DA) and joint learning-based DA (JL-DA). Experiments on a Korean speech database demonstrate that the verification performance is improved significantly in realworld environments by using self-adaptive soft VAD.

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

“…The pooled vector is passed to one or few fully-connected (FC) layers to generate the deep speaker embedding z. The works in [14], [19], [20], [33] are examples of this approach.…”

“…To improve the robustness of the SV model to long nonspeech segments, we proposed self-adaptive soft VAD (SAS-VAD) [33], which is the combination of soft VAD and selfadaptive VAD. Here, we introduce the advanced version of SAS-VAD which shows better performance than the original one and can be combined with the MSA to achieve our ultimate goal.…”

“…In this subsection, we explain our previous soft VAD [33] first and its advanced version later. Unlike typical VADs that make a hard decision on acoustic features with a predefined threshold, the soft VAD makes a soft decision on speaker feature vectors when the self-attentive pooling (SAP) [30] is applied.…”

“…(2) They contain audio recordings which already have small portion of non-speech. However, our previous work [33] shows the need of the robust VAD for SV in real-world environments, where the input audio contains long non-speech segments in noisy and reverberant environments. In these adverse environments, the energy-based VAD produces unreliable speech frames, which degrades the performance of SV systems [34].…”

“…To satisfy these two requirements for TI-SV, which is our ultimate goal in this paper, we present our methods: feature pyramid module (FPM)-based multi-scale aggregation (MSA) [22] and self-adaptive soft VAD (SAS-VAD) [33]. We employ the FPM-based MSA to deal with short speech segments.…”