Supervised Determined Source Separation with Multichannel Variational Autoencoder

Kameoka, Hirokazu; Li, Li; Inoue, Shota; Makino, Shoji

doi:10.1162/neco_a_01217

Cited by 91 publications

(71 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VAEs have recently been studied for modeling and generation of speech signals [28,29,30], and synthesizing music sounds in [31]. They have also been used for speech enhancement [32,33] and feature learning for ASR [34,35]. Recent studies in ASV have studied the use of VAEs in data augmentation [36], regularisation [37] and domain adaptation [38] for deep speaker embeddings (x-vectors).…”

Section: Introductionmentioning

confidence: 99%

Deep generative variational autoencoding for replay spoof detection in automatic speaker verification

Chettri

Kinnunen

Benetos

2020

Computer Speech & Language

View full text Add to dashboard Cite

Automatic speaker verification (ASV) systems are highly vulnerable to presentation attacks, also called spoofing attacks. Replay is among the simplest attacks to mount -yet difficult to detect reliably. The generalization failure of spoofing countermeasures (CMs) has driven the community to study various alternative deep learning CMs. The majority of them are supervised approaches that learn a human-spoof discriminator. In this paper, we advocate a different, deep generative approach that leverages from powerful unsupervised manifold learning in classification. The potential benefits include the possibility to sample new data, and to obtain insights to the latent features of genuine and spoofed speech. To this end, we propose to use variational autoencoders (VAEs) as an alternative backend for replay attack detection, via three alternative models that differ in their class-conditioning. The first one, similar to the use of Gaussian mixture models (GMMs) in spoof detection, is to train independently two VAEs -one for each class. The second one is to train a single conditional model (C-VAE) by injecting a one-hot class label vector to the encoder and decoder networks. Our final proposal integrates an auxiliary classifier to guide the learning of the latent space. Our experimental results using constant-Q cepstral coefficient (CQCC) features on the ASVspoof 2017 and 2019 physical access subtask datasets indicate that the C-VAE offers substantial improvement in comparison to training two separate VAEs for each class. On the 2019 dataset, the C-VAE outperforms the VAE and the baseline GMM by an absolute 9 -10% in both equal error rate (EER) and tandem detection cost function (t-DCF) metrics. Finally, we propose VAE residuals -the absolute difference of the original input and the reconstruction as features for spoofing detection. The proposed frontend approach augmented with a convolutional neural network classifier demonstrated substantial improvement over the VAE backend use case.

show abstract

Section: Introductionmentioning

confidence: 99%

Deep generative variational autoencoding for replay spoof detection in automatic speaker verification

Chettri

Kinnunen

Benetos

2020

Computer Speech & Language

View full text Add to dashboard Cite

show abstract

“…In the experiments, we will compare this variational E-step with an alternative proposed in [29], which consists in relying only on a point estimate of the latent variables. In our framework, this approach can be understood as assuming that the approximate posterior q(z x; θenc) is a dirac delta function centered at the maximum a posteriori estimate z ⋆ .…”

Section: Point-estimate E-stepmentioning

confidence: 99%

A Recurrent Variational Autoencoder for Speech Enhancement

Leglaive

Alameda-Pineda

Girin

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This paper presents a generative approach to speech enhancement based on a recurrent variational autoencoder (RVAE). The deep generative speech model is trained using clean speech signals only, and it is combined with a nonnegative matrix factorization noise model for speech enhancement. We propose a variational expectationmaximization algorithm where the encoder of the RVAE is finetuned at test time, to approximate the distribution of the latent variables given the noisy speech observations. Compared with previous approaches based on feed-forward fully-connected architectures, the proposed recurrent deep generative speech model induces a posterior temporal dynamic over the latent variables, which is shown to improve the speech enhancement results.Index Terms-Speech enhancement, recurrent variational autoencoders, nonnegative matrix factorization, variational inference.

show abstract

“…both single-channel [8][9][10] and multichannel scenarios [11][12][13][14][15]. The main idea of these studies is to use variational autoencoders (VAEs) [16] to replace the NMF generative model, thus benefiting from DNNs' representational power.…”

Section: Introductionmentioning

confidence: 99%

“…Once the VAE is trained, the encoder could in principle be used to approximate the true posterior. Interestingly, [15] proposes a heuristic algorithm for multichannel source separation based on [13] which uses this property of the encoder to achieve computational efficiency. However this inference algorithm is not statistically principled.…”

Section: Introductionmentioning

confidence: 99%

A Statistically Principled and Computationally Efficient Approach to Speech Enhancement Using Variational Autoencoders

Pariente¹,

Deleforge²

2019

Interspeech 2019

View full text Add to dashboard Cite

Recent studies have explored the use of deep generative models of speech spectra based of variational autoencoders (VAEs), combined with unsupervised noise models, to perform speech enhancement. These studies developed iterative algorithms involving either Gibbs sampling or gradient descent at each step, making them computationally expensive. This paper proposes a variational inference method to iteratively estimate the power spectrogram of the clean speech. Our main contribution is the analytical derivation of the variational steps in which the encoder of the pre-learned VAE can be used to estimate the variational approximation of the true posterior distribution, using the very same assumption made to train VAEs. Experiments show that the proposed method produces results on par with the aforementioned iterative methods using sampling, while decreasing the computational cost by a factor 36 to reach a given performance.

show abstract

Supervised Determined Source Separation with Multichannel Variational Autoencoder

Cited by 91 publications

References 36 publications

Deep generative variational autoencoding for replay spoof detection in automatic speaker verification

Deep generative variational autoencoding for replay spoof detection in automatic speaker verification

A Recurrent Variational Autoencoder for Speech Enhancement

A Statistically Principled and Computationally Efficient Approach to Speech Enhancement Using Variational Autoencoders

Contact Info

Product

Resources

About