Nonintrusive Speech Intelligibility Prediction Using Convolutional Neural Networks

Abstract

Speech Intelligibility Prediction (SIP) algorithms are becoming popular tools within the development and operation of speech processing devices and algorithms. However, many SIP algorithms require knowledge of the underlying clean speech; a signal that is often not available in real-world applications. This has led to increased interest in non-intrusive SIP algorithms, which do not require clean speech to make predictions. In this paper we investigate the use of Convolutional Neural Networks (CNNs) for non-intrusive SIP. To do so, we utilize a CNN architecture that shows similarities to existing SIP algorithms, in terms of computational structure, and which allows for easy and meaningful visualization and interpretation of trained weights. We evaluate this architecture using a large dataset obtained by combining datasets from the literature. The proposed method shows high prediction performance when compared with four existing intrusive and non-intrusive SIP algorithms. This demonstrates the potential of deep learning for speech intelligibility prediction.

N-MTTL SI Model: Non-Intrusive Multi-Task Transfer Learning-Based Speech Intelligibility Prediction Model with Scenery Classification

“…Kendall's (τ) values obtained by the N-MTTL SI model (ResNet-18 MT) are 0.80 and 0.69 for seen and unseen conditions, respectively. Overall, the accuracy measures of the model are comparable to the literature [17], where similar values are observed for unseen conditions. However, conditions in our dataset are not identical to [17], therefore direct comparison is not possible.…”

“…The residual block also contains a batch normalization layer and Rectified Linear Unit (ReLU) activation function, which is used after every convolutional layer [26]. Previous work [17] has explored the importance of the convolution layer to extract spectro-temporal patterns in the input signal related to speech intelligibility. Therefore, we expect the convolutional layers of ResNet to be beneficial for both our specific tasks in our N-MTTL SI model.…”

“…Spearman rank correlation (ρ) confirms monotonicity between the estimated and labelled intelligibility, which is 0.93/0.85 for the seen/unseen conditions, respectively. Lastly, Kendall's rank correlation coefficient, as used in the speech intelligibility prediction literature [17,18], also expresses the degree of monotonicity in the relation between measurements and predictions. Kendall's (τ) values obtained by the N-MTTL SI model (ResNet-18 MT) are 0.80 and 0.69 for seen and unseen conditions, respectively.…”

“…Recently, rapid progress in the deep learning domain has also been utilised in the field of speech intelligibility prediction. Several models have been introduced employing DNN [16], CNN [17,18] or U-Net [19]. However, none of the speech intelligibility models proposed so far provide any additional information regarding the cause of the speech intelligibility degradation that could be further used to fine-tune speech enhancement algorithms.…”

A Non-Intrusive Speech Intelligibility Estimation Method Based on Deep Learning Using Autoencoder Features

“…However, since there is no reference signal for the receiver such as communication, a non-intrusive intelligibility estimation method is required. In [3], the speech intelligibility is predicted using convolutional neural network which is trained with measured intelligibility scores that humans listen and evaluate. The work in [4] presented the method of speech intelligibility prediction by using automatic speech recognition (ASR) system based deep neural networks.…”