Double Multi-Head Attention for Speaker Verification

Abstract

Most state-of-the-art Deep Learning systems for text-independent speaker verification are based on speaker embedding extractors. These architectures are commonly composed of a feature extractor front-end together with a pooling layer to encode variable-length utterances into fixed-length speaker vectors. In this paper we present Double Multi-Head Attention (MHA) pooling, which extends our previous approach based on Self MHA. An additional self attention layer is added to the pooling layer that summarizes the context vectors produced by MHA into a unique speaker representation. This method enhances the pooling mechanism by giving weights to the information captured for each head and it results in creating more discriminative speaker embeddings. We have evaluated our approach with the VoxCeleb2 dataset. Our results show 6.09% and 5.23% relative improvement in terms of EER compared to Self Attention pooling and Self MHA, respectively. According to the obtained results, Double MHA has shown to be an excellent approach to efficiently select the most relevant features captured by the CNN-based front-ends from the speech signal.

Speaker Characterization by means of Attention Pooling

“…The results of [17] experiments are shown in Table 1. Performance was evaluated using Equal Error Rate (EER).…”

“…The DMHSA system has been assessed in [17] by VoxCeleb dataset [20,21]. VoxCeleb is a large multimedia database that contains more than one million 16kHz audio utterances for more than 6K celebrities and has two different versions with several evaluation protocols.…”

“…VoxCeleb is a large multimedia database that contains more than one million 16kHz audio utterances for more than 6K celebrities and has two different versions with several evaluation protocols. For [17] experiments, Vox-Celeb2 development partition with no augmentation has been used to train all models. The performance of these systems has been evaluated with Vox1-Test, Vox1-E, and Vox1-H conditions.…”

“…In [17], DMHSA Pooling was proposed. It is a DL system with an attention-based pooling layer that was developed for SV.…”

“…The authors have recently proposed a DL system based on a Double Multi-Head Self-Attention (DMHSA) pooling [17]. Its architecture consists of a Convolutional Neural Network (CNN)-based front-end, followed by an attention-based pooling layer and a set of fully connected layers.…”