An Evaluation of Intrusive Instrumental Intelligibility Metrics

Abstract

Instrumental intelligibility metrics are commonly used as an alternative to listening tests. This paper evaluates 12 monaural intrusive intelligibility metrics: SII, HEGP, CSII, HASPI, NCM, QSTI, STOI, ESTOI, MIKNN, SIMI, SIIB, and sEPSM corr . In addition, this paper investigates the ability of intelligibility metrics to generalize to new types of distortions and analyzes why the top performing metrics have high performance. The intelligibility data were obtained from 11 listening tests described in the literature. The stimuli included Dutch, Danish, and English speech that was distorted by additive noise, reverberation, competing talkers, pre-processing enhancement, and post-processing enhancement. SIIB and HASPI had the highest performance achieving a correlation with listening test scores on average of ρ = 0.92 and ρ = 0.89, respectively. The high performance of SIIB may, in part, be the result of SIIBs developers having access to all the intelligibility data considered in the evaluation. The results show that intelligibility metrics tend to perform poorly on data sets that were not used during their development. By modifying the original implementations of SIIB and STOI, the advantage of reducing statistical dependencies between input features is demonstrated. Additionally, the paper presents a new version of SIIB called SIIB Gauss , which has similar performance to SIIB and HASPI, but takes less time to compute by two orders of magnitude.

Intelligibility and Listening Effort of Spanish Oesophageal Speech

“…Objective measurements, on the other hand, include measures like Speech Transmission Index (STI), Short Term Objective Intelligibility (STOI) [9], and also ASR. Newer approaches of intelligibility measurement can be found in [10][11][12].…”

Predicting the effects of periodicity on the intelligibility of masked speech: An evaluation of different modelling approaches and their limitations

“…Moreover, the materials often vary considerably, both across models, as well as regarding their acoustic properties across testing conditions, which makes it difficult to compare different models and understand their shortcomings. Consequently, testing a set of existing speech intelligibility models with a common data set obtained with materials that systematically vary with respect to certain relevant acoustic features appears to be a fruitful approach to compare, challenge, and further improve them (for a similar approach, see Schubotz et al, 2016, andVan Kuyk et al, 2018). At the same time, modelling experimental data also serves to gain a better understanding of them, particularly by examining the internal signal representations generated by models with different theoretical assumptions.…”

Vocal Effort Based Speaking Style Conversion Using Vocoder Features and Parallel Learning

“…Instead intelligibility of our system was studied using a recently developed instrumental intelligibility metric called speech intelligibility in bits (SIIB, [69]). This measure is based on the mutual information between a clean reference and a noisy signal, and it performed well in a recent survey that compared several instrumental methods for measuring speech intelligibility [70]. In the current study, SIIB Gauss [70], a variation of SIIB that uses the information capacity of a Gaussian channel for mutual information calculation, was used.…”