A CHiME-3 challenge system: Long-term acoustic features for noise robust automatic speech recognition

Abstract

The paper describes an automatic speech recognition (ASR) system for the 3rd CHiME challenge that addresses noisy acoustic scenes within public environments. The proposed system includes a multi-channel speech enhancement front-end including a microphone channel failure detection method that is based on cross-comparing the modulation spectra of speech to detect erroneous microphone recordings. The main focus of the submission is the investigation of the amplitude modulation filter bank (AMFB) as a method to extract long-term acoustic cues prior to a Gaussian mixture model (GMM) or deep neural network (DNN) based ASR classifier. It is shown that AMFB features outperform the commonly used frame splicing technique of filter bank features even on a performance optimized ASR challenge system. I.e., temporal analysis of speech by hand-crafted and auditory motivated AMFBs is shown to be more robust compared to a data-driven method based on extracting temporal dynamics with a DNN

An analysis of environment, microphone and data simulation mismatches in robust speech recognition

“…The improvement brought by these techniques appears to be quite correlated between real and simulated data. Other authors also found this result to hold for auditory-motivated features such as Gabor filterbank (GBFB) (Martinez and Meyer, 2015) and amplitude modulation filter bank (AMFB) (Moritz et al, 2015) and feature transformation/augmentation methods such as vocal tract length normalization (VTLN) (Tachioka et al, 2015) or i-vectors (Pang and Zhu, 2015;Prudnikov et al, 2015), provided that these features and methods are applied to noisy data or data enhanced using the robust beamforming or source separation techniques listed in Section 3.2. Interestingly, Tachioka et al (2015) found VTLN to yield consistent results on real vs. simulated data when using GEV beamforming as a pre-processing step but opposite results when using MVDR beamforming instead.…”

The third ‘CHiME’ speech separation and recognition challenge: Dataset, task and baselines

“…[20,35] use a Gammatone filterbank that has broader filter tails and has been shown to provide noise robustness. Four systems have used amplitude modulation-based features either by applying a discrete cosine transform (DCT) on the filterbank envelopes [37]; employing a 2D Gabor filter bank [22]; or tracking amplitude modulation (AM) in filterbands using a non-linear Teager energy operator [19].…”

The third ‘CHiME’ speech separation and recognition challenge: Analysis and outcomes

“…Several teams have combined multiple architectures (Yoshioka et al, 2015;Du et al, 2015;Zhuang et al, 2015). Performance benefits of the various architectures remain unclear, however it is notable that some of the best scoring systems including Hori et al (2015), Sivasankaran et al (2015) and Moritz et al (2015), have used the baseline DNN configuration.…”

“…Examples include, Ma et al (2015) and Du et al (2015) that use a Gammatone filterbank that has broader filter tails and has been shown to perform well in previous robust ASR evaluations. Four systems used amplitude modulationbased features either by applying a discrete cosine transform (DCT) on the filterbank envelopes (Castro Martinez and Meyer, 2015); employing a 2D Gabor filter bank (Moritz et al, 2015); or tracking amplitude modulation (AM) in filterbands using a non-linear Teager energy operator (Hori et al, 2015).…”

“…The simplest approach has been to apply utterance-based feature mean and variance normalization (Zhao et al, 2015;Fujita et al, 2015;Du et al, 2015;Wang et al, 2015). However, the two most effective techniques are transforming the DNN features using feature-space maximum likelihood linear regression (fMLLR) (Hori et al, 2015;Moritz et al, 2015;Vu et al, 2015;Sivasankaran et al, 2015;Tran et al, unpublished) or augmentation of the DNN features using either i-vectors, (e.g., Moritz et al, 2015;Zhuang et al, 2015), pitch-based features (Ma et al, 2015;Wang et al, 2015;Du et al, 2015) or bottleneck features (Tachioka et al, 2015), i.e., extracted from bottleneck layers in speaker classification DNNs. Where i-vectors have been used they may be either per-speaker (e.g., Prudnikov et al, 2015) or per-speaker-environment, (e.g.…”