A simple model of speech communication and its application to intelligibility enhancement

Kleijn, Willem; Hendriks, RC

doi:10.26686/wgtn.14337164.v1

Cited by 3 publications

(13 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jensen and Taal (2014) and Taghia and Martin (2014) hypothesised that the intelligibility of a degraded speech signal is related to the mutual information between the clean and degraded speech signal. Kleijn and Hendriks (2015) and Khademi et al (2017) developed speech enhancement algorithms based on information theory. The research in this thesis can be viewed as an extension of those ideas.…”

Section: Approaches To Modelling Speech Communicationmentioning

confidence: 99%

“…The idea that talker variability could be modelled as noise was taken up again over 50 years later in Kleijn and Hendriks (2015), which coined the term production noise. It was shown, theoretically, that production noise would cause the effectiveness of communication to saturate at a particular bit rate.…”

Section: An Information Theoretical Perspective Of Talker Variabilitymentioning

confidence: 99%

“…Thus, unlike (2.28), even if the SNR of the environmental channel is infinite, the information rate of speech communication is finite. Kleijn and Hendriks (2015) and Khademi et al (2017) used this concept to design a state-of-the art speech enhancement algorithm. The enhancement algorithm redistributes power across frequency bands using a variant of the water-filling algorithm (Cover and Thomas, 2012).…”

Section: An Information Theoretical Perspective Of Talker Variabilitymentioning

confidence: 99%

“…Furthermore, note that (3.10) and (3.11) satisfy the Markov condition in (3.1). This additive Gaussian model was first proposed as a model of speech communication in Kleijn and Hendriks (2015), where it was used to develop an algorithm to increase the intelligibility of speech degraded by additive environmental noise.…”

Section: The Univariate Gaussian Channelmentioning

confidence: 99%

“…When Kleijn and Hendriks (2015) proposed (3.10) and (3.11) as a model of speech communication, a third equation was included:…”

Section: Interpretation Noisementioning

confidence: 99%

See 4 more Smart Citations

Speech Communication from an Information Theoretical Perspective

Kuyk¹

View full text Add to dashboard Cite

Throughout the last century, models of human speech communication have been proposed by linguists, psychologists, and engineers. Advancements have been made, but a theory of human speech communication that is both comprehensive and quantitative is yet to emerge. This thesis hypothesises that a branch of mathematics known as information theory holds the answer to a more complete theory. Information theory has made fundamental contributions to wireless communications, computer science, statistical inference, cryptography, thermodynamics, and biology. There is no reason that information theory cannot be applied to human speech communication, but thus far, a relatively small effort has been made to do so. The goal of this research was to develop a quantitative model of speech communication that is consistent with our knowledge of linguistics and that is accurate enough to predict the intelligibility of speech signals. Specifically, this thesis focuses on the following research questions: 1) how does the acoustic information rate of speech compare to the lexical information rate of speech? 2) How can information theory be used to predict the intelligibility of speech-based communication systems? 3) How well do competing models of speech communication predict intelligibility? To answer the first research question, novel approaches for estimating the information rate of speech communication are proposed. Unlike existing approaches, the methods proposed in this thesis rely on having a chorus of speech signals where each signal in the chorus contains the same linguistic message, but is spoken by a different talker. The advantage of this approach is that variability inherent in the production of speech can be accounted for. The approach gives an estimate of about 180 b/s. This is three times larger than estimates based on lexical models, but it is an order of magnitude smaller than previous estimates that rely on acoustic signals. To answer the second research question, a novel instrumental intelligibility metric called speech intelligibility in bits (SIIB) and a variant called SIIBGauss are proposed. SIIB is an estimate of the amount of information shared between a talker and a listener in bits per second. Unlike existing intelligibility metrics that are based on information theory, SIIB accounts for talker variability and statistical dependencies between time-frequency units. Finally, to answer the third research question, a comprehensive evaluation of intrusive intelligibility metrics is provided. The results show that SIIB and SIIBGauss have state-of-the-art performance, that intelligibility metrics tend to perform poorly on data sets that were not used during their development, and show the advantage of reducing statistical dependencies between input features.

show abstract

Section: Approaches To Modelling Speech Communicationmentioning

confidence: 99%

Section: An Information Theoretical Perspective Of Talker Variabilitymentioning

confidence: 99%

Section: An Information Theoretical Perspective Of Talker Variabilitymentioning

confidence: 99%

Section: The Univariate Gaussian Channelmentioning

confidence: 99%

“…When Kleijn and Hendriks (2015) proposed (3.10) and (3.11) as a model of speech communication, a third equation was included:…”

Section: Interpretation Noisementioning

confidence: 99%

See 3 more Smart Citations

Speech Communication from an Information Theoretical Perspective

Kuyk¹

View full text Add to dashboard Cite

show abstract

iMetricGAN: Intelligibility Enhancement for Speech-in-Noise Using Generative Adversarial Network-Based Metric Learning

Li¹,

Fu²,

Yu³

et al. 2020

Interspeech 2020

View full text Add to dashboard Cite

The intelligibility of natural speech is seriously degraded when exposed to adverse noisy environments. In this work, we propose a deep learning-based speech modification method to compensate for the intelligibility loss, with the constraint that the root mean square (RMS) level and duration of the speech signal are maintained before and after modifications. Specifically, we utilize an iMetricGAN approach to optimize the speech intelligibility metrics with generative adversarial networks (GANs). Experimental results show that the proposed iMetricGAN outperforms conventional state-of-the-art algorithms in terms of objective measures, i.e., speech intelligibility in bits (SIIB) and extended short-time objective intelligibility (ESTOI), under a Cafeteria noise condition. In addition, formal listening tests reveal significant intelligibility gains when both noise and reverberation exist.

show abstract

Mutual information based speech intelligibility prediction and its application to hearing aid fitting

Wang¹

View full text Add to dashboard Cite

Speech enhancement was, is, and will be the key technology for digital speech transmission. When developing speech enhancement algorithms, the intelligibility or the quality of the processed speech needs to be evaluated. Intelligibility is more fundamental than quality. The evaluation of speech intelligibility can be carried out through subjective listening tests and objective metrics. Carrying out subjective listening tests is time-consuming and costly. Using the objective metrics is more efficient. Thus, there has been an increasing research in speech intelligibility prediction. Speech intelligibility can be measured in terms of the information received by a listener. This thesis aims at developing a mutual information-based speech intelligibility predictor (SIP) and using the mutual information-based predictor to assist the fitting process in hearing instruments. To achieve this goal, this thesis carried out three studies. First, it studied the modeling of the transmitted message. For mutual information-based SIPs, the most important thing is to determine the transmitted message. This thesis studied two approaches of modeling of the message: one is the continuous-valued sound, and the other is the discrete-valued linguistic message. Two corresponding SIPs were developed. By comparing their predicted intelligibility results with the psychometric curves, which are the subjective intelligibility scores, it shows that the modeling of discrete-valued message gives a better match to the psychometric curves. Second, based on the modeling of the discrete-valued message, this thesis proposed a mutual information-based SIP. Since the discrete-valued message cannot be obtained from a speech signal, the proposed SIP calculates the mutual information between the clean speech and the received speech, instead of calculating the mutual information between the message and the received speech. The proposed SIP considers frequency correlation for both the clean speech and the received speech. The evaluation results show that the proposed SIP performs better than the existing state-of-the-art mutual information-based SIPs. Third, this thesis proposed an automatic fitting tool for the nonlinear frequency compression (NFC) operator, which is a frequency lowering operator used in hearing instruments. The automatic fitting tool adjusts the parameter in the NFC by maximizing the mutual information between the message and the frequency-lowered speech. To evaluate the automatic fitting tool, the parameter was also searched by listening tests. The results show that the parameter determined by the automatic fitting tool is consistent to the parameter determined by the listening tests.

show abstract

A simple model of speech communication and its application to intelligibility enhancement

Cited by 3 publications

References 0 publications

Speech Communication from an Information Theoretical Perspective

Speech Communication from an Information Theoretical Perspective

iMetricGAN: Intelligibility Enhancement for Speech-in-Noise Using Generative Adversarial Network-Based Metric Learning

Mutual information based speech intelligibility prediction and its application to hearing aid fitting

Contact Info

Product

Resources

About