Creating Clarity in Noisy Environments by Using Deep Learning in Hearing Aids

Abstract

Hearing aids continue to acquire increasingly sophisticated sound-processing features beyond basic amplification. On the one hand, these have the potential to add user benefit and allow for personalization. On the other hand, if such features are to benefit according to their potential, they require clinicians to be acquainted with both the underlying technologies and the specific fitting handles made available by the individual hearing aid manufacturers. Ensuring benefit from hearing aids in typical daily listening environments requires that the hearing aids handle sounds that interfere with communication, generically referred to as “noise.” With this aim, considerable efforts from both academia and industry have led to increasingly advanced algorithms that handle noise, typically using the principles of directional processing and postfiltering. This article provides an overview of the techniques used for noise reduction in modern hearing aids. First, classical techniques are covered as they are used in modern hearing aids. The discussion then shifts to how deep learning, a subfield of artificial intelligence, provides a radically different way of solving the noise problem. Finally, the results of several experiments are used to showcase the benefits of recent algorithmic advances in terms of signal-to-noise ratio, speech intelligibility, selective attention, and listening effort.

Restoring speech intelligibility for hearing aid users with deep learning

“…While we do not test for it here, we also expect improvements in cognitive load with our system, considering that it increases the effective SNR of noisy sounds by up to 16 dB 45 . In contrast to studies using traditional denoising algorithms, a recent study has shown that deep learning based denoising 23 on hearing aids can improve speech intelligibility for hearing impaired subjects. Our work is in line with other research in this field 16 – 18 and shows the improvements to speech intelligibility that can be made with more powerful neural network models.…”

“…For the majority of hearing aid users, with less severe hearing loss 22 , it is more challenging to provide intelligibility improvements through denoising. A very recent study has shown the ability of a deep learning based denoising system to moderately improve speech intelligibility for hearing aid users 23 . Our work builds upon these exciting results and demonstrates that deep learning based denoising may be used to provide large improvements in speech intelligibility for hearing aid users in the near future.…”

Speech recognition with a hearing-aid processing scheme combining beamforming with mask-informed speech enhancement

“…Further research might also examine performance in conditions in which the beamformer is not optimised for the true target position, since this optimisation may limit the potential for mask-informed enhancement to provide additional benefit. Finally, it would also be of interest to compare the performance of the processing methods used here to that of other approaches to integrating spatial filtering and speech enhancement (e.g., Andersen et al, 2021;Koutrouvelis, Hendriks, Heusdens & Jensen, 2017).…”

“…These include the potential for bilateral HAs to exchange information and the use of deep neural networks (DNNs) to aid the separation of speech from noise (e.g., Kolbaek, Yu, Tan & Jensen, 2017;Luo & Mesgarani, 2019;Xu, Du, Dai & Lee, 2015). A major focus of current research concerns how to integrate techniques such as spatial filtering and speech enhancement (e.g., Andersen et al, 2021;Koutrouvelis, Hendriks, Heusdens & Jensen, 2017). The ultimate goal is to raise the signal-to-noise-ratio (SNR) of target speech as much as possible while maintaining sufficient flexibility to accommodate changes in the target source and preserving a natural seeming sound environment, including spatial information.…”

The Audible Contrast Threshold (ACT™) test: a clinical spectro-temporal modulation detection test

“…However, individuals with a sensorineural hearing loss often experience great difficulties in understanding speech in noisy environments (Kochkin, 2000), also referred to as “supra-threshold distortion”, which is not necessarily well predicted from pure-tone audiometry (Vermiglio and Fang, 2022) and which amplification alone does not solve (Plomp, 1978; Grant et al, 2013). Most of today’s state-of-the-art hearing aids have the ability to improve speech understanding in challenging speech-in-noise situations, but the need for optimal fitting of these sound-processing features vary greatly among individuals (Andersen et al, 2021; Zaar et al, 2023b). To fit the hearing aids such that each patient’s individual needs are met, the clinician needs information about the degree to which the patient is struggling in real life speech-in-noise scenarios, when audibility is compensated for.…”