Non-intrusive intelligibility prediction using a codebook-based approach

Abstract

Abstract-It could be beneficial for users of hearing aids if these were able to automatically adjust the processing according to the speech intelligibility in the specific acoustic environment. Most speech intelligibility metrics are intrusive, i.e., they require a clean reference signal, which is rarely available in real-life applications. This paper proposes a method, which allows using an intrusive short-time objective intelligibility (STOI) metric without requiring access to a clean signal. The clean speech reference signal is replaced by the clean speech envelope spectrum estimated from the noisy signal. The spectral envelope has been shown to be an important cue for speech intelligibility and is used as the reference signal inside STOI. The spectral envelopes are estimated as a combination of predefined dictionaries, i.e., codebooks, that best fits the noisy speech signal. The simulations show a high correlation between the proposed non-intrusive codebookbased STOI (NIC-STOI) and the intrusive STOI indicating that NIC-STOI is a suitable metric for automatic classification of speech signals.

Validation of the Non-Intrusive Codebook-Based Short Time Objective Intelligibility Metric for Processed Speech

“…It can be observed that NIC-STOI is highly correlated with the subjective scores across all noise conditions. Furthermore, NIC-STOI is also highly correlated with STOI, which supports the earlier findings in [14,15]. The highest deviation can be observed for the SSN noise condition, which can perhaps be explained by the noise codebook weighting this condition less when being trained on all the noise conditions concatenated.…”

“…In order to further validate the performance of the NIC-STOI metric presented in [14,15] we here evaluate it on the same data as in the original paper on STOI [16,5]. Subjective intelligibility scores have been obtained from 15 normal hearing subjects.…”

“…The NIC-STOI metric, proposed in [14,15], is based on STOI but does not require access to a clean reference signal. Figure 1 depicts an overview of the NIC-STOI algorithm.…”

“…where Θ denotes the support space to be estimated. Figure 1: A block diagram of the NIC-STOI metric adapted from [14,15]. Using a codebook-based approach the speech and noisy spectra are jointly modeled from pre-trained codebooks of both speech and noise and are then used within STOI.…”

“…For instance, machine learning [8,9], noise reduction [10,11], principal component analysis [12] and neural network [13] methods have been proposed as approaches to obtain a reference signal to use inside the STOI framework from the degraded speech signal. Another non-intrusive version of the STOI metric, the non-intrusive codebook-based STOI (NIC-STOI), is proposed in [14,15]. This is based on estimating the spectrum of the reference signal from its degraded version by identifying combinations of pre-trained codebook entries of speech and noise spectra, parametrized by Auto-Regressive (AR) parameters, which best fit the degraded speech signal.…”