Incorporating Real-world Noisy Speech in Neural-network-based Speech Enhancement Systems

Abstract

Supervised speech enhancement relies on parallel databases of degraded speech signals and their clean reference signals during training. This setting prohibits the use of real-world degraded speech data that may better represent the scenarios where such systems are used. In this paper, we explore methods that enable supervised speech enhancement systems to train on real-world degraded speech data. Specifically, we propose a semi-supervised approach for speech enhancement in which we first train a modified vector-quantized variational autoencoder that solves a source separation task. We then use this trained autoencoder to further train an enhancement network using real-world noisy speech data by computing a triplet-based unsupervised loss function. Experiments show promising results for incorporating real-world data in training speech enhancement systems.

Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation

“…Due to the data acquisition challenge mentioned above, it is desirable to leverage real conversational data without any supervision signals. For single-channel speech enhancement and separation tasks, various techniques to leverage unsupervised data have been investigated, including cycle-consistencyloss-based training [16,17], semi-supervised training [18,19], WavLM [20], and mixture invariant training [21]. However, no work has been done to leverage real conversational unsupervised data for the multi-channel speech separation modeling, where the model must learn the nonlinear correlation between the input channels while solving the permutation problem of the multiple output signals.…”