Speech enhancement for robust automatic speech recognition: Evaluation using a baseline system and instrumental measures

Abstract

Automatic speech recognition in everyday environments must be robust to significant levels of reverberation and noise. One strategy to achieve such robustness is multi-microphone speech enhancement. In this study, we present results of an evaluation of different speech enhancement pipelines using a state-of-the-art ASR system for a wide range of reverberation and noise conditions. The evaluation exploits the recently released ACE Challenge database which includes measured multichannel acoustic impulse responses from 7 different rooms with reverberation times ranging from 0.33 s to 1.34 s. The reverberant speech is mixed with ambient, fan and babble noise recordings made with the same microphone setups in each of the rooms. In the first experiment performance of the ASR without speech processing is evaluated. Results clearly indicate the deleterious effect of both noise and reverberation. In the second experiment, different speech enhancement pipelines are evaluated with relative word error rate reductions of up to 82%. Finally, the ability of selected instrumental metrics to predict ASR performance improvement is assessed. The best performing metric, Short-Time Objective Intelligibility Measure, is shown to have a Pearson correlation coefficient of 0.79, suggesting that it is a useful predictor of algorithm performance in these tests

A Cross-Entropy-Guided (CEG) Measure for Speech Enhancement Front-End Assessing Performances of Back-End Automatic Speech Recognition

“…Please note that the correlation score between the CEG and WER for multi-condition training tends to be smaller than that for clean-condition training because the number of the same scores of WER corresponding to different scores of CEG for multi-condition training is larger, which leads to worse correlation statistics. Unlike the conclusions in [10,11], we observe that the correlation scores between the STOI and WER tend to be larger than those between the PESQ and WER for multi-condition training and the contrary conclusion could be drawn for clean-condition training, which may indicate that the recognition performance depends more largely on speech quality for clean-condition training and on speech intelligibility for multi-condition training. Besides, it is noted that the acoustic confidence measure and the proposed CEG have positive correlations with WER, on the contrary, PESQ and STOI have negative correlations with WER.…”

“…For example, resulting speech enhanced by OM-LSA could improve recognition accuracy for cleancondition training regardless of its worse STOI shown in Table 2. Accordingly, the conclusion in [10,11] that the correlation coefficient between the WER and STOI is higher than other distortion measures (e.g., PESQ) is not accurate and reliable enough. Some researches [22,32] suggested by the conclusion in [10,11] designed a speech enhancement frontend to especially improve STOI and thus achieve better ASR performance.…”

“…In [4][5][6][7], a very good correlation between WER and the perceptual evaluation of speech quality (PESQ) [8] for measuring speech quality has been verified. Recently, it has been shown that the correlation coefficient between WER and the short-time objective intelligibility (STOI) [9] for measuring speech intelligibility is higher than other distortion measures [10,11]. Besides, in [12,13], an acoustic confidence measure usually defined as the entropy of the posterior distribution from the output of the artificial neural network (ANN) for the acoustic model was proposed and shown a high degree of correlation with WER, where the discriminatory power of the ANN decreases and the posterior probabilities tend to become more uniform with a higher entropy.…”

Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition

“…Recent studies have reported a positive correlation between objective intelligibility scores and ASR performance [27,32]. In Table 2, we show the STOI and PESQ scores of enhanced speech processed by RLSE1 and RLSE2 at SNR levels of0 and 5 dB.…”

“…It has been reported that when the goal is to improve the ASR performance, ideal binary mask (IBM) is more suitable than ideal ratio mask (IRM) or directly mapping [27] to be used to design the SE system. Therefore, we implement an IBM-based SE system in this study.…”