Speech Enhancement with Stochastic Temporal Convolutional Networks

Richter, Julius; Carbajal, Guillaume; Gerkmann, Timo

doi:10.21437/interspeech.2020-2588

Cited by 18 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, we showed that we could improve the performance even with a small amount of noisy-clean speech data. For future work, our approach could also be integrated with deep generative models that combine temporal dependencies [27].…”

Section: Discussionmentioning

confidence: 99%

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

Fang

Carbajal

Wermter

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Recently, a generative variational autoencoder (VAE) has been proposed for speech enhancement to model speech statistics. However, this approach only uses clean speech in the training phase, making the estimation particularly sensitive to noise presence, especially in low signal-to-noise ratios (SNRs). To increase the robustness of the VAE, we propose to include noise information in the training phase by using a noise-aware encoder trained on noisy-clean speech pairs. We evaluate our approach on real recordings of different noisy environments and acoustic conditions using two different noise datasets. We show that our proposed noise-aware VAE outperforms the standard VAE in terms of overall distortion without increasing the number of model parameters. At the same time, we demonstrate that our model is capable of generalizing to unseen noise conditions better than a supervised feedforward deep neural network (DNN). Furthermore, we demonstrate the robustness of the model performance to a reduction of the noisy-clean speech training data size.

show abstract

Section: Discussionmentioning

confidence: 99%

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

Fang

Carbajal

Wermter

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Improving the classifier by taking time dependencies and/or visual information into account could further improve the guided VAE. The model could then be compared to other deep generative models taking temporal dependencies into account [23].…”

Section: Discussionmentioning

confidence: 99%

Guided Variational Autoencoder for Speech Enhancement with a Supervised Classifier

Carbajal

Richter

Gerkmann

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Recently, variational autoencoders have been successfully used to learn a probabilistic prior over speech signals, which is then used to perform speech enhancement. However, variational autoencoders are trained on clean speech only, which results in a limited ability of extracting the speech signal from noisy speech compared to supervised approaches. In this paper, we propose to guide the variational autoencoder with a supervised classifier separately trained on noisy speech. The estimated label is a high-level categorical variable describing the speech signal (e.g. speech activity) allowing for a more informed latent distribution compared to the standard variational autoencoder. We evaluate our method with different types of labels on real recordings of different noisy environments. Provided that the label better informs the latent distribution and that the classifier achieves good performance, the proposed approach outperforms the standard variational autoencoder and a conventional neural networkbased supervised approach.

show abstract

“…Very few studies deal with the inherent limitation of the VAE to handle sequential data, that is sequences of samples that are not statistically independent, as is the case of speech data. To the best of our knowledge, only [28] and [37] proposed generative approaches to speech enhancement based on VAE variants that can learn temporal dependencies in the speech model. While [28] proposed a recurrent VAE (RVAE) based on standard recurrent neural networks (RNNs), [37] used stochastic temporal convolutional network (TCNs) [38], [39], allowing the latent variables to have both hierarchical and temporal dependencies.…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders

Bie,

Leglaive,

Alameda-Pineda

et al. 2021

Preprint

View full text Add to dashboard Cite

Dynamical variational auto-encoders (DVAEs) are a class of deep generative models with latent variables, dedicated to time series data modeling. DVAEs can be considered as extensions of the variational autoencoder (VAE) that include the modeling of temporal dependencies between successive observed and/or latent vectors in data sequences. Previous work has shown the interest of DVAEs and their better performance over the VAE for speech signals (spectrogram) modeling. Independently, the VAE has been successfully applied to speech enhancement in noise, in an unsupervised noise-agnostic set-up that does not require the use of a parallel dataset of clean and noisy speech samples for training, but only requires clean speech signals. In this paper, we extend those works to DVAE-based single-channel unsupervised speech enhancement, hence exploiting both speech signals unsupervised representation learning and dynamics modeling. We propose an unsupervised speech enhancement algorithm based on the most general form of DVAEs, that we then adapt to three specific DVAE models to illustrate the versatility of the framework. More precisely, we combine DVAE-based speech priors with a noise model based on nonnegative matrix factorization, and we derive a variational expectation-maximization (VEM) algorithm to perform speech enhancement. Experimental results show that the proposed approach based on DVAEs outperforms its VAE counterpart and a supervised speech enhancement baseline.

show abstract

Speech Enhancement with Stochastic Temporal Convolutional Networks

Cited by 18 publications

References 19 publications

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

Variational Autoencoder for Speech Enhancement with a Noise-Aware Encoder

Guided Variational Autoencoder for Speech Enhancement with a Supervised Classifier

Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoders

Contact Info

Product

Resources

About