Frequency and Temporal Convolutional Attention for Text-Independent Speaker Recognition

Abstract

Majority of the recent approaches for text-independent speaker recognition apply attention or similar techniques for aggregation of frame-level feature descriptors generated by a deep neural network (DNN) front-end. In this paper, we propose methods of convolutional attention for independently modelling temporal and frequency information in a convolutional neural network (CNN) based front-end. Our system utilizes convolutional block attention modules (CBAMs) [1] appropriately modified to accommodate spectrogram inputs. The proposed CNN front-end fitted with the proposed convolutional attention modules outperform the no-attention and spatial-CBAM baselines by a significant margin on the Vox-Celeb [2, 3] speaker verification benchmark. Our best model achieves an equal error rate of 2.031% on the VoxCeleb1 test set, which is a considerable improvement over comparable state of the art results. For a more thorough assessment of the effects of frequency and temporal attention in real-world conditions, we conduct ablation experiments by randomly dropping frequency bins and temporal frames from the input spectrograms, concluding that instead of modelling either of the entities, simultaneously modelling temporal and frequency attention translates to better real-world performance.

A Multi-View Approach to Audio-Visual Speaker Verification

“…In [7], thin ResNet-34 architecture is used along with GhostVLAD pooling that results in 3.2% EER. In [8], a convolutional attention model is proposed for time and frequency dimensions and GhostVLAD based aggregation is applied and the model achieves 2.0%. The lowest EER on the VoxCeleb1 test set is reported in [13], where their best system makes use of data augmentation and system combination.…”

“…In Table 1, we present the EERs on VC1 and VC2 datasets. The upper part of the table includes audio-only (A-only) EER of [8,13] and attention based fusion proposed in [3]. In the lower part of the table, the first two rows show the unimodal performance of our systems.…”

“…A-only of [13] NA 1.0 A-only of [8] NA 2.0 AV of [3] 5.3 NA Our unimodal A-only 3.5 2.2 Our unimodal V-only 3.4 3.9 Mid-level AV fusion 2.0 1.4 Score fusion unimodal A+V 1.7 0.9 Score fusion A+V+AV 1.6 0.7 Table 1: EER (%) of various models on VC1 and VC2 test sets ror cases. First, there is a small percentage (estimate 1%) of the dataset which is labelled incorrectly.…”

“…In recent studies, neural network based speaker embeddings, such as xvectors [6], are used. These systems usually process the input speech by a network that generates a sequence of features for the utterance which are then aggregated into a single vector to represent the speaker embedding [7,8]. These aggregation or summarization methods range from temporal pooling to cluster based approaches which are also used in computer vision studies such as NetVLAD [9] and GhostVLAD [10,7].…”

Short-Time Spectral Aggregation for Speaker Embedding

“…There are also embedding systems that use ResNets [10] or DenseNets [11] for frame-level processing. For example, lightweight ResNets were adapted from ResNet-34 and ResNet-50 in [12] and [13]. In [5], a deep embedding network was implemented based on DenseNets.…”