Boosting Noise Robustness of Acoustic Model via Deep Adversarial Training

Abstract

In realistic environments, speech is usually interfered by various noise and reverberation, which dramatically degrades the performance of automatic speech recognition (ASR) systems. To alleviate this issue, the commonest way is to use a well-designed speech enhancement approach as the front-end of ASR. However, more complex pipelines, more computations and even higher hardware costs (microphone array) are additionally consumed for this kind of methods. In addition, speech enhancement would result in speech distortions and mismatches to training. In this paper, we propose an adversarial training method to directly boost noise robustness of acoustic model. Specifically, a jointly compositional scheme of generative adversarial net (GAN) and neural network-based acoustic model (AM) is used in the training phase. GAN is used to generate clean feature representations from noisy features by the guidance of a discriminator that tries to distinguish between the true clean signals and generated signals. The joint optimization of generator, discriminator and AM concentrates the strengths of both GAN and AM for speech recognition. Systematic experiments on CHiME-4 show that the proposed method significantly improves the noise robustness of AM and achieves the average relative error rate reduction of 23.38% and 11.54% on the development and test set, respectively.

UFANS: U-Shaped Fully-Parallel Acoustic Neural Structure for Statistical Parametric Speech Synthesis

“…Within U Net, those pooling and up-sampling operation along the spatial dimension make the receptive field increases exponentially and high way connection allows the combination of different scales features. Inspired by the success of U Net in image segmentation, U-shaped models have been proposed for various acoustic applications, e.g., denoise [14], audio source separation [15].…”

Investigation of Monaural Front-End Processing for Robust Speech Recognition Without Retraining or Joint-Training

“…The first one is using a separation frontend to enhance both training and test sets and retraining the acoustic model with enhanced features [12,13]. The second one is joint-training the front-end enhancement model with the back-end acoustic model [14,15]. The third one is multi-conditional training which performs acoustic modeling on noisy speech and the extracted features are directly fed to the acoustic model for decoding at the test stage.…”