A Deep Neural Network for Short-Segment Speaker Recognition

Hajavi, Amirhossein; Etemad, Ali

doi:10.21437/interspeech.2019-2240

Cited by 58 publications

(57 citation statements)

References 51 publications

(121 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For all the pooling methods mentioned above, we use only single-scale feature map from the last layer of the feature extractor. Recently, multi-scale aggregation (MSA) methods have been proposed to exploit speaker information at multiple time scales [22], [23], [48], [49], showing the effectiveness in dealing with variable-duration test utterances.…”

Section: Deep Speaker Embedding Learningmentioning

confidence: 99%

“…Even with this robustness, using multi-scale features from multiple layers (Fig. 2(b)), called multi-scale aggregation (MSA), has shown better performance than using singlescale feature maps [22], [23], [48], [49]. Note that, between the frame-and segment-level operations, we should choose the segment-level operation for the MSA because all the feature maps from different layers have the same time scale in the frame-level operation.…”

Section: Multi-scale Aggregationmentioning

confidence: 99%

“…The concatenated vector was fed into fully-connected layers to generate the speaker embedding. Hajavi et al [23] proposed a similar approach using UtterIdNet to deal with short speech segments. They showed that the MSA is useful for short-duration speaker verification by extracting as much information as possible from short speech segments.…”

Section: A Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

et al. 2020

View full text Add to dashboard Cite

Speaker verification (SV) has recently attracted considerable research interest due to the growing popularity of virtual assistants. At the same time, there is an increasing requirement for an SV system: it should be robust to short speech segments, especially in noisy and reverberant environments. In this paper, we consider one more important requirement for practical applications: the system should be robust to an audio stream containing long non-speech segments, where a voice activity detection (VAD) is not applied. To meet these two requirements, we introduce feature pyramid module (FPM)-based multiscale aggregation (MSA) and self-adaptive soft VAD (SAS-VAD). We present the FPM-based MSA to deal with short speech segments in noisy and reverberant environments. Also, we use the SAS-VAD to increase the robustness to long non-speech segments. To further improve the robustness to acoustic distortions (i.e., noise and reverberation), we apply a masking-based speech enhancement (SE) method. We combine SV, VAD, and SE models in a unified deep learning framework and jointly train the entire network in an endto-end manner. To the best of our knowledge, this is the first work combining these three models in a deep learning framework. We conduct experiments on Korean indoor (KID) and VoxCeleb datasets, which are corrupted by noise and reverberation. The results show that the proposed method is effective for SV in the challenging conditions and performs better than the baseline i-vector and deep speaker embedding systems.

show abstract

Section: Deep Speaker Embedding Learningmentioning

confidence: 99%

Section: Multi-scale Aggregationmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

et al. 2020

View full text Add to dashboard Cite

show abstract

“…2-D CNNs have also shown competitive results for speaker verification. There are Computer Vision architectures such as VGG [10,7,11,9] and ResNet [8,12,13] that have been adapted to capture speaker discriminative information from the Mel-Spectrograms. In fact, Resnet34 has shown a better performance than TDNN in the most recent speaker verification challenges [14,15].…”

Section: Introductionmentioning

confidence: 99%

Double Multi-Head Attention for Speaker Verification

India

Safari

Hernando

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Most state-of-the-art Deep Learning systems for text-independent speaker verification are based on speaker embedding extractors. These architectures are commonly composed of a feature extractor front-end together with a pooling layer to encode variable-length utterances into fixed-length speaker vectors. In this paper we present Double Multi-Head Attention (MHA) pooling, which extends our previous approach based on Self MHA. An additional self attention layer is added to the pooling layer that summarizes the context vectors produced by MHA into a unique speaker representation. This method enhances the pooling mechanism by giving weights to the information captured for each head and it results in creating more discriminative speaker embeddings. We have evaluated our approach with the VoxCeleb2 dataset. Our results show 6.09% and 5.23% relative improvement in terms of EER compared to Self Attention pooling and Self MHA, respectively. According to the obtained results, Double MHA has shown to be an excellent approach to efficiently select the most relevant features captured by the CNN-based front-ends from the speech signal.

show abstract

“…[5] proposed the statistics pooling layer, which combines mean and standard deviation statistics for weighted aggregation of temporal frames. More recently, [6] proposed time-distributed voting (TDV) for aggregating features extracted by their UtterId-Net front-end in short segment speaker verification, especially sub-second durations. [7] proposed the usage of dictionarybased NetVLAD or GhostVLAD [8] for aggregating temporal features, using a 34-layer ResNet based front-end for feature extraction.…”

Section: Introductionmentioning

confidence: 99%

Frequency and Temporal Convolutional Attention for Text-Independent Speaker Recognition

Yadav¹,

Rai²

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Majority of the recent approaches for text-independent speaker recognition apply attention or similar techniques for aggregation of frame-level feature descriptors generated by a deep neural network (DNN) front-end. In this paper, we propose methods of convolutional attention for independently modelling temporal and frequency information in a convolutional neural network (CNN) based front-end. Our system utilizes convolutional block attention modules (CBAMs) [1] appropriately modified to accommodate spectrogram inputs. The proposed CNN front-end fitted with the proposed convolutional attention modules outperform the no-attention and spatial-CBAM baselines by a significant margin on the Vox-Celeb [2, 3] speaker verification benchmark. Our best model achieves an equal error rate of 2.031% on the VoxCeleb1 test set, which is a considerable improvement over comparable state of the art results. For a more thorough assessment of the effects of frequency and temporal attention in real-world conditions, we conduct ablation experiments by randomly dropping frequency bins and temporal frames from the input spectrograms, concluding that instead of modelling either of the entities, simultaneously modelling temporal and frequency attention translates to better real-world performance.

show abstract

A Deep Neural Network for Short-Segment Speaker Recognition

Cited by 58 publications

References 51 publications

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

A Unified Deep Learning Framework for Short-Duration Speaker Verification in Adverse Environments

Double Multi-Head Attention for Speaker Verification

Frequency and Temporal Convolutional Attention for Text-Independent Speaker Recognition

Contact Info

Product

Resources

About