Non-intrusive codebook-based intelligibility prediction

Sørensen, Charlotte Mehlin; Kavalekalam, Mathew Shaji; Xenaki, Angeliki; Boldt, Jesper B.; Christensen, Mads Græsbøll

doi:10.1016/j.specom.2018.06.003

Cited by 8 publications

(3 citation statements)

References 39 publications

(86 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, the automatic classification of acoustic environments in which speech enhancement would benefit a HA user is vital [4,5]. In principle, this could be achieved by an objective speech intelligibility prediction metric running online in HAs [6].…”

Section: Introductionmentioning

confidence: 99%

“…The aforementioned objective models are intrusivethey require a reference signal to determine predictions of speech intelligibility, which is not available in real-world scenarios. On the contrary, non-intrusive speech intelligibility models (ModA [14], SRMR [15], NIC-STOI [6]) require only a degraded speech signal for speech intelligibility prediction. This advantage makes non-intrusive models more suitable for real-world applications [5].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

N-MTTL SI Model: Non-Intrusive Multi-Task Transfer Learning-Based Speech Intelligibility Prediction Model with Scenery Classification

Marcinek¹,

Stone²,

Millman³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

The application of speech enhancement algorithms for hearing aids may not always be beneficial to increasing speech intelligibility. Therefore, a prior environment classification could be important. However, previous speech intelligibility models do not provide any additional information regarding the reason for a decrease in speech intelligibility. We propose a unique non-intrusive multi-task transfer learning-based speech intelligibility prediction model with scenery classification (N-MTTL SI model). The solution combines a Mel-spectrogram analysis of the degraded speech signal with transfer learning and multi-task learning to provide simultaneous speech intelligibility prediction (task 1) and scenery classification of ten real-world noise conditions (task 2). The model utilises a pre-trained ResNet architecture as an encoder for feature extraction. The prediction accuracy of the N-MTTL SI model for both tasks is high. Specifically, RMSE of speech intelligibility predictions for seen and unseen conditions is 3.76% and 4.06%. The classification accuracy is 98%. In addition, the proposed solution demonstrates the potential of using pre-trained deep learning models in the domain of speech intelligibility prediction.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

N-MTTL SI Model: Non-Intrusive Multi-Task Transfer Learning-Based Speech Intelligibility Prediction Model with Scenery Classification

Marcinek¹,

Stone²,

Millman³

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

“…Among these measures the articulation index [3], the speech transmission index (STI) [4], the speech intelligibility index (SII) [5], the short-time objective intelligibility (STOI) [6] and mutual-information-based techniques, such as the algorithm proposed in [7], are intrusive. Non-intrusive measures include a non-intrusive extension of the STOI [8], [9], that relies on estimating the amplitude envelope of the clean speech from the input signal, and measures relying on machine learning techniques. Some measures use a trained speech recognizer as proposed in [10], [11] or a neural network trained to predict SI from a sequence of spectral features [12].…”

Section: Introductionmentioning

confidence: 99%

Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete Latent Representations

McKinney,

Cauchi

2021

Preprint

View full text Add to dashboard Cite

Non-intrusive speech intelligibility (SI) prediction from binaural signals is useful in many applications. However, most existing signal-based measures are designed to be applied to single-channel signals. Measures specifically designed to take into account the binaural properties of the signal are often intrusive -characterised by requiring access to a clean speech signaland typically rely on combining both channels into a singlechannel signal before making predictions. This paper proposes a non-intrusive SI measure that computes features from a binaural input signal using a combination of vector quantization (VQ) and contrastive predictive coding (CPC) methods. VQ-CPC feature extraction does not rely on any model of the auditory system and is instead trained to maximise the mutual information between the input signal and output features. The computed VQ-CPC features are input to a predicting function parameterized by a neural network. Two predicting functions are considered in this paper. Both feature extractor and predicting functions are trained on simulated binaural signals with isotropic noise. They are tested on simulated signals with isotropic and real noise. For all signals, the ground truth scores are the (intrusive) deterministic binaural STOI. Results are presented in terms of correlations and MSE and demonstrate that VQ-CPC features are able to capture information relevant to modelling SI and outperform all the considered benchmarks -even when evaluating on data comprising of different noise field types.

show abstract

Nonintrusive objective measurement of speech intelligibility: A review of methodology

Feng

Chen

2022

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

Non-intrusive codebook-based intelligibility prediction

Cited by 8 publications

References 39 publications

N-MTTL SI Model: Non-Intrusive Multi-Task Transfer Learning-Based Speech Intelligibility Prediction Model with Scenery Classification

N-MTTL SI Model: Non-Intrusive Multi-Task Transfer Learning-Based Speech Intelligibility Prediction Model with Scenery Classification

Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete Latent Representations

Nonintrusive objective measurement of speech intelligibility: A review of methodology

Contact Info

Product

Resources

About