Fusion of multiple uncertainty estimators and propagators for noise robust ASR

Trung, Dung Tran; Jouvet, Denis

doi:10.1109/icassp.2014.6854657

Cited by 13 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During acoustic model scoring, the uncertainty decoding framework estimates the uncertainty (or variance) of speech distortion in the input features [2][3][4] in each time frame and modifies the acoustic scores accordingly. The uncertainty can be computed directly in the ASR feature domain [1,[5][6][7][8][9][10] or propagated from the spectral domain to the feature domain [11][12][13][14][15][16][17], under the assumption that it can be represented by Gaussian distribution. For Gaussian mixture model (GMM) based acoustic models, the expectation of the acoustic scores over this distribution can then be computed in closed form by adding the variance of the uncertainty to that of every Gaussian component [2][3][4].…”

Section: Introductionmentioning

confidence: 99%

Consistent DNN uncertainty training and decoding for robust ASR

Nathwani

Illina

2017

2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Self Cite

View full text Add to dashboard Cite

We consider the problem of robust automatic speech recognition (ASR) in noisy conditions. The performance improvement brought by speech enhancement is often limited by residual distortions of the enhanced features, which can be seen as a form of statistical uncertainty. Uncertainty estimation and propagation methods have recently been proposed to improve the ASR performance with deep neural network (DNN) acoustic models. However, the performance is still limited due to the use of uncertainty only during decoding. In this paper, we propose a consistent approach to account for uncertainty in the enhanced features during both training and decoding. We estimate the variance of the distortions using a DNN uncertainty estimator that operates directly in the feature maximum likelihood linear regression (fMLLR) domain and we then sample the uncertain features using the unscented transform (UT). We report the resulting ASR performance on the CHiME-2 and CHiME-3 datasets for different uncertainty estimation/propagation techniques. The proposed DNN uncertainty training method brings 4% and 8% relative improvement on these two datasets, respectively, compared to a competitive fMLLR-domain DNN acoustic modeling baseline.

show abstract

Section: Introductionmentioning

confidence: 99%

Consistent DNN uncertainty training and decoding for robust ASR

Nathwani

Illina

2017

2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Conversely, the features are unreliable when their uncertainty tends to be high. The uncertainty is first computed in the spectral domain [4] then propagated into the feature domain. Because of the non-linear transform applied to the input spectral domain, propagation requires approximate methods such as Vector Taylor series (VTS) [5], moment matching [6] or unscented transform [7].…”

Section: Introductionmentioning

confidence: 99%

“…To overcome this, the estimated uncertainty can be rescaled by a linear transformation [4,[8][9][10]. In the past [4,10], the scaling factors were optimized such that the recaled uncertainty estimates are close to the oracle estimates irrespectively of the resulting state hypotheses. This can be considered as a sub-optimal approach because the same scaling factors are applied to the correct state hypothesis and to the competing state hypotheses.…”

Section: Introductionmentioning

confidence: 99%

“…Recently, maximum mutual information (MMI) [11] and boosted MMI (bMMI) [12,13] were successfully employed for supervised discriminative adaptation of feature means and diagonal uncertainty matrices [1]. In this approach, the diagonal uncertainty matrix was estimated directly in the feature domain as the squared difference between noisy and enhanced features, which was shown to be a poor estimate of uncertainty [4].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Discriminative uncertainty estimation for noise robust ASR

Trung

Jouvet

2015

2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We consider the problem of uncertainty estimation for noiserobust ASR. Existing uncertainty estimation techniques improve ASR accuracy but they still exhibit a gap compared to the use of oracle uncertainty. This comes partly from the highly non-linear feature transformation and from additional assumptions such as Gaussian distribution and independence between frequency bins in the spectral domain. In this paper, we propose a method to rescale the estimated feature-domain full uncertainty covariance matrix in a statedependent fashion according to a discriminative criterion. The state-dependent and feature index-dependent scaling factors are learned from development data. Experimental evaluation on Track 1 of the 2nd CHiME challenge data shows that discriminative rescaling leads to better results than generative rescaling. Moreover, discriminative rescaling of the Wiener uncertainty estimator leads to 12% relative word error rate reduction compared to discriminative rescaling of the alternative estimator in [1].

show abstract

“…The distribution of speech distortions is typically approximated as a Gaussian from which the uncertainty or variance of speech distortions is derived. The uncertainty can be computed directly in the ASR feature domain [2,[6][7][8][9][10] or propagated from the spectral domain to the feature domain [1,[11][12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

An extended experimental investigation of DNN uncertainty propagation for noise robust ASR

Nathwani

Morales-Cordovilla

Sivasankaran

et al. 2017

2017 Hands-Free Speech Communications and Microphone Arrays (HSCMA)

Self Cite

View full text Add to dashboard Cite

Automatic speech recognition (ASR) in noisy environments remains a challenging goal. Recently, the idea of estimating the uncertainty about the features obtained after speech enhancement and propagating it to dynamically adapt deep neural network (DNN) based acoustic models has raised some interest. However, the results in the literature were reported on simulated noisy datasets for a limited variety of uncertainty estimators. We found that they vary significantly in different conditions. Hence, the main contribution of this work is to assess DNN uncertainty decoding performance for different data conditions and different uncertainty estimation/propagation techniques. In addition, we propose a neural network based uncertainty estimator and compare it with other uncertainty estimators. We report detailed ASR results on the CHiME-2 and CHiME-3 datasets. We find that, on average, uncertainty propagation provides similar relative improvement on real and simulated data and that the proposed uncertainty estimator performs significantly better than the one in [1]. We also find that the improvement is consistent, but it depends on the signal-to-noise ratio (SNR) and the noise environment.

show abstract

Fusion of multiple uncertainty estimators and propagators for noise robust ASR

Cited by 13 publications

References 22 publications

Consistent DNN uncertainty training and decoding for robust ASR

Consistent DNN uncertainty training and decoding for robust ASR

Discriminative uncertainty estimation for noise robust ASR

An extended experimental investigation of DNN uncertainty propagation for noise robust ASR

Contact Info

Product

Resources

About