Restricted Boltzmann machines for vector representation of speech in speaker recognition

Ghahabi, Omid; Hernando, Javier

doi:10.1016/j.csl.2017.06.007

Cited by 37 publications

(18 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A first attempt to use RBMs at the backend in a speaker verification task was made in Reference [20]. The authors put their efforts into the front end of a speaker verification system, in order to learn a compact and fixed dimensional speaker representation in the form of a speaker vector by means of RBM adaptation [21][22][23][24]. They also make use of DBNs in the i-vector/PLDA (Probabilistic Linear Discriminant Analysis) framework for speaker verification at the backend [25].…”

Section: Introductionmentioning

confidence: 99%

Restricted Boltzmann Machine Vectors for Speaker Clustering and Tracking Tasks in TV Broadcast Shows

2019

Self Cite

View full text Add to dashboard Cite

Restricted Boltzmann Machines (RBMs) have shown success in both the front-end and backend of speaker verification systems. In this paper, we propose applying RBMs to the front-end for the tasks of speaker clustering and speaker tracking in TV broadcast shows. RBMs are trained to transform utterances into a vector based representation. Because of the lack of data for a test speaker, we propose RBM adaptation to a global model. First, the global model—which is referred to as universal RBM—is trained with all the available background data. Then an adapted RBM model is trained with the data of each test speaker. The visible to hidden weight matrices of the adapted models are concatenated along with the bias vectors and are whitened to generate the vector representation of speakers. These vectors, referred to as RBM vectors, were shown to preserve speaker-specific information and are used in the tasks of speaker clustering and speaker tracking. The evaluation was performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed speaker clustering system gained up to 12% relative improvement, in terms of Equal Impurity (EI), over the baseline system. On the other hand, in the task of speaker tracking, our system has a relative improvement of 11% and 7% compared to the baseline system using cosine and Probabilistic Linear Discriminant Analysis (PLDA) scoring, respectively.

show abstract

Section: Introductionmentioning

confidence: 99%

Restricted Boltzmann Machine Vectors for Speaker Clustering and Tracking Tasks in TV Broadcast Shows

2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…In [22,23,24], several attempts have been made to improve the performance of speaker verification, using unsupervised learning such as Restricted Boltzmann Machines (RBMs) and Deep Belief Networks (DBNs). As an example of frontend, in [22,25], a vector representation of speakers was proposed by means of RBM adaptation.…”

Section: Introductionmentioning

confidence: 99%

Auto-Encoding Nearest Neighbor i-Vectors for Speaker Verification

Khan¹,

India²,

Hernando³

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

In the last years, i-vectors followed by cosine or PLDA scoring techniques were the state-of-the-art approach in speaker verification. PLDA requires labeled background data, and there exists a significant performance gap between the two scoring techniques. In this work, we propose to reduce this gap by using an autoencoder to transform i-vector into a new speaker vector representation, which will be referred to as ae-vector. The autoencoder will be trained to reconstruct neighbor i-vectors instead of the same training i-vectors, as usual. These neighbor i-vectors will be selected in an unsupervised manner according to the highest cosine scores to the training i-vectors. The evaluation is performed on the speaker verification trials of VoxCeleb-1 database. The experiments show that our proposed ae-vectors gain a relative improvement of 42% in terms of EER compared to the conventional i-vectors using cosine scoring, which fills the performance gap between cosine and PLDA scoring techniques by 92%, but without using speaker labels.

show abstract

“…The proposed embeddings are alternatives to traditional i-vectors [18] and the recent x-vectors [19] with much lower computational cost and suitable for online processing. The proposed embedding network is a DNN taking advantage of input speaker-corrupted supervectors and efficient variable ReLU (VReLU) [20] as an activation function trying to discriminate the background speakers. Speaker corruption is performed by adding supervectors, built by only 20 speech frames randomly selected from other speakers, to the supervectors of a given speaker.…”

Section: Introductionmentioning

confidence: 99%

Speaker-Corrupted Embeddings for Online Speaker Diarization

Ghahabi

Fischer

2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Speaker diarization is more challenging in presence of background noise or music, frequent speaker changes, and cross talks. In an online scenario, the decision should be made at time, given only the current short segment and the speakers detected in the past, which makes the task even harder. In this work, an online robust speaker diarization algorithm is proposed in which speech segments are represented by low dimensional vectors referred to as speaker-corrupted embeddings. The proposed speaker embedding network is a deep neural network which takes speaker-corrupted supervectors as input, uses variable ReLU (VReLU) as an activation function, and tries to discriminate the background speakers. Speaker corruption is performed by adding supervectors built by 20 speech frames from other speakers to the supervectors of a given speaker. It is shown that speaker corruption, VReLU, and input dropout increase the generalization power of the proposed network. To increase the robustness, the proposed embeddings are concatenated with LDA transformed supervectors. Experimental results on the Albayzin 2018 evaluation set show a competitive accuracy, more robustness, and much lower computational cost compared to typical offline algorithms.

show abstract

Restricted Boltzmann machines for vector representation of speech in speaker recognition

Cited by 37 publications

References 10 publications

Restricted Boltzmann Machine Vectors for Speaker Clustering and Tracking Tasks in TV Broadcast Shows

Restricted Boltzmann Machine Vectors for Speaker Clustering and Tracking Tasks in TV Broadcast Shows

Auto-Encoding Nearest Neighbor i-Vectors for Speaker Verification

Speaker-Corrupted Embeddings for Online Speaker Diarization

Contact Info

Product

Resources

About