Non-intrusive Speech Intelligibility Metric Prediction for Hearing Impaired Individuals

Abstract

This paper proposes neural models to predict Speech Intelligibility (SI),both by prediction of established SI metrics and of human speech recognition (HSR) on the 1st Clarity Prediction Challenge. Both intrusive and non-intrusive predictors for intrusive SI metrics are trained, then fine-tuned on the HSR ground truth. Results are reported on a number of SI metrics, and the model choice for the Clarity challenge submission is explained. Additionally, the relationship between the SI scores in the data and commonly used signal processing metrics which approximate SI are analysed, and some issues emerging from this relationship discussed. It is found that intrusive neural predictors of SI metrics when fine-tuned on the true HSR scores outperform the non neural challenge baseline.

Multi-CMGAN+/+: Leveraging Multi-Objective Speech Quality Metric Prediction for Speech Enhancement

“…Recent work in speech enhancement [6,21,22] have found that the outputs of HuBERT's encoder stage H FE (•) are particularly useful for capturing quality-related information, outperforming the final transformer layer and weighted sums of each transformer output. The outputs of H FE (•) are 2D representations with dimensions 512 × T where T depends on the length of the input audio in seconds.…”

Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users Using Intermediate ASR Features and Human Memory Models

“…The best performing non-intrusive approach [12] uses an uncertainty measure derived from state-of-theart ASR systems as a proxy for human intelligibility, finding a strong correlation between the two measures. Other successful approaches [13,14] make use of powerful feature representations derived from self-supervised speech representations (SSSRs) as inputs to neural speech intelligibility prediction models, while others use neural network structures which have been shown to be useful in the related task of human speech quality rating prediction [15]. CPC2 differs from CPC1 in that its evaluation sets are disjoint in terms of listener and hearing aid system relative to its training sets.…”

“…A model structure following work on the CPC1 in [14] is chosen for the primary SI prediction network (cf. Section 3.2.1), depicted to the right in Figure 2.…”

“…This model structure has previously been successfully applied to the task of human quality label prediction [19] using pre-trained SSSR representations as its input feature. This approach was also found to be useful for the CPC1 task [14], however generalization to unseen hearing aid systems was poor.…”