Temporal contrast normalization and edge-preserved smoothing on temporal modulation structure for robust speech recognition

Abstract

In this paper, we propose a two-step processing algorithm which adaptively normalizes the temporal modulation of speech to extract robust speech feature for automatic speech recognition systems. The first step processing is to normalize the temporal modulation contrast (TMC) of the cepstral time series for both clean and noisy speech. The second step processing is to smooth the normalized temporal modulation structure to reduce the artifacts due to noise while preserving the speech modulation events (edges). We tested our algorithm on speech recognition experiments in additive noise condition (AURORA-2J data corpus), reverberant noise condition (convolution of clean speech utterances from AURORA-2J with a smart room impulse response), and noisy condition with both reverberant and additive noise (air conditioner noise in a smart room). For comparison, the ETSI advanced front-end (AFE) algorithm was used. Our results showed that the algorithm provided: (1) for additive noise condition, 57.26% relative word error reduction (RWER) rate for clean conditional training (59.37% for AFE), and 33.52% RWER rate for multi-conditional training (35.77% for AFE), (2) for reverberant condition, 51.28% RWER rate (10.17% for AFE) and (3) for noisy condition with both reverberant and additive noise, 71.74% RWER rate (48.86% for AFE).

Temporal Modulation Normalization for Robust Speech Feature Extraction and Recognition

“…The recognition performance based on the normalized cepstral coefficients were improved [6]. In this section, we further test our proposed method for noisy speech recognition.…”

“…In this section, we first give a brief introduction of the noise effect on the temporal modulation of speech, then present our proposed method that is used to reduce the noise effect [6].…”

“…In addition, most of the ASR systems use the cepstral coefficients as features. Therefore, we investigated the noise effect on the temporal modulation of cepstral coefficients [6] (more specifically temporal trajectories of the cepstral coefficients). We found that in the cepstral domain, the temporal modulation contrast (TMC) were systematically reduced in noisy environments, and the modulation spectrum of the cepstral temporal trajectory is similarly as the low-pass filtered one of the clean speech.…”

“…Some studies show that the essential speech intelligibility information is mostly encoded in the temporal modulations of the acoustic signals [2], [3], [4], [5]. Inspired by this knowledge, we have investigated the noise effect on the temporal modulation of speech in either additive noise or reverberant noise environments, and proposed a noise attenuation method by assuming that the temporal modulation structures of speech and noise are different [6]. Although the temporal structure normalization (TSN) has been proposed for robust speech recognition [7], [8], it was based on normalizing both the modulation of clean and noisy speech to a reference modulation spectrum which was calculated as an average of a training clean data set.…”

“…This paper reports a continuous work of the study in [6]. We mainly focus on examining the proposed method for ASR.…”