From Features to Speaker Vectors by means of Restricted Boltzmann Machine Adaptation

Abstract

Restricted Boltzmann Machines (RBMs) have shown success in different stages of speaker recognition systems. In this paper, we propose a novel framework to produce a vector-based representation for each speaker, which will be referred to as RBMvector. This new approach maps the speaker spectral features to a single fixed-dimensional vector carrying speaker-specific information. In this work, a global model, referred to as Universal RBM (URBM), is trained taking advantage of RBM unsupervised learning capabilities. Then, this URBM is adapted to the data of each speaker in the development, enrolment and evaluation datasets. The network connection weights of the adapted RBMs are further concatenated and subject to a whitening with dimension reduction stage to build the speaker vectors. The evaluation is performed on the core test condition of the NIST SRE 2006 database, and it is shown that RBM-vectors achieve 15% relative improvement in terms of EER compared to i-vectors using cosine scoring. The score fusion with i-vector attains more than 24% relative improvement. The interest of this result for score fusion yields on the fact that both vectors are produced in an unsupervised fashion and can be used instead of i-vector/PLDA approach, when no data label is available. Results obtained for RBM-vector/PLDA framework is comparable with the ones from i-vector/PLDA. Their score fusion achieves 14% relative improvement compared to i-vector/PLDA.

Self Multi-Head Attention for Speaker Recognition

“…Speaker embedding is often referred to a single low dimensional vector representation of the speaker characteristics from a speech signal extracted using a Neural Network (NN) model. For textindependent Speaker Recognition (SR), which is the focus of this work, these models can be trained either in a supervised (e.g., [1,2]) or in an unsupervised (e.g., [3,4]) fashion. Supervised speaker embeddings are produced by training a deep architecture using speaker-labeled background data.…”

Restricted Boltzmann Machine Vectors for Speaker Clustering

“…They also make use of DBNs at the backend in the i-vector framework for speaker verification [10]. As a continuation to these works, a successful attempt was made in our previous work to apply RBMs as a front-end for learning a fixed dimensional speaker representation [11]. This vector representation of speaker was referred to as RBM vector.…”

“…This vector representation of speaker was referred to as RBM vector. In [11] it has been shown that the RBM vector preserves speaker specific information and has shown com-petitive results as compared to the conventional i-vector based speaker verification systems. This has lead us to apply the RBM vector for learning speaker representation in the task of speaker clustering.…”

“…In this work we consider the use of RBMs for vector representation of speakers. We extend the use of RBM vector [11] in the context of speaker clustering. First, we extract RBM vectors for all the speaker utterances in the same way as in [11].…”

“…We extend the use of RBM vector [11] in the context of speaker clustering. First, we extract RBM vectors for all the speaker utterances in the same way as in [11]. Then, we perform a bottom-up AHC clustering for all the RBM vectors using cosine or Probabilistic Linear Discriminant Analysis (PLDA) scores.…”