On the Interaction of Head and Gaze Control With Acoustic Beam Width of a Simulated Beamformer in a Two-Talker Scenario

Abstract

Superdirectional acoustic beamforming technology provides a high signal-to-noise ratio, but potential speech intelligibility benefits to hearing aid users are limited by the way the users move their heads. Steering the beamformer using eye gaze instead of head orientation could mitigate this problem. This study investigated the intelligibility of target speech with a dynamically changing direction when heard through gaze-controlled (GAZE) or head-controlled (HEAD) superdirectional simulated beamformers. The beamformer provided frequency-independent noise attenuation of either 8 dB (WIDE [moderately directional]) or 12 dB (NARROW [highly directional]) relative to no beamformer referred as the OMNI (omni-directional) condition. Before the main experiment, signal-to-noise ratios were normalized for each participant and each beam width condition to yield equal percentage of correct performance in a reference condition. Hence, results are presented as normalized speech intelligibility (NSI). In an ongoing presentation, the participants ( n = 17), of varying degree of hearing loss, heard single-word targets every 1.5 s coming from either left (−30°) or right (+30°) presented in continuous, spatially distributed, speech-shaped noise. When the target was static, NSI was better in the GAZE than in the HEAD condition, but only when the beam was NARROW. When the target switched location without warning, NSI performance dropped. In this case, the WIDE HEAD condition provided the best average NSI performance, because some participants tended to orient their head in between the targets, allowing them to hear out the target regardless of location. The difference in NSI between GAZE and HEAD conditions for individual participants was related to the observed head-orientation strategy, which varied widely across participants.

Eye-gaze Estimation with HEOG and Neck EMG using Deep Neural Networks

“…In another study [13], an eye-gaze steering HA was found to be able to improve the speech intelligibility for HI listeners compared to non-steering one. Similar results could also be found in other studies [11], [15].…”

“…Therefore, when the auditory attended target switches between talkers, the listener would saccade and rotate head to direct the eye-gaze accordingly [7], [8]. Based on these studies, eye-gaze was treated as an indicator of the attended target and used to steer HAs [9]- [15]. For example, an eye-tracker-based eye-gaze selection of auditory target was reported to outperform the button-pressing-based manual selection, for both the measurement of target recalling and switching time [9].…”

“…This is likely because the fitted model was fixed and the authors assumed it applied to data of all trials. However, in face-to-face conversations, listeners usually eye-gaze at the target talker but not always with the head orienting to it [7], [15], which indicates the strategy of both saccade and head-rotation vary in realistic scenarios. These results suggested that eye-gaze estimation methods adopting the hand-crafted features or fixed-form regression function were lacked of generalization.…”

Self-motion with Hearing Impairment and (Directional) Hearing Aids

“…This could indicate that the participant groups had a similar gaze-related self-motion, or that the measures are not sensitive enough to detect differences. In accordance with the results of Lu et al (2021) and Hládek et al (2019) , the participants used a combination of head and eye movements to look at the target. The significant differences in HeadGazeRatio that were found between the NH and HI participant groups indicate that the HI participants covered a larger part of the movement with head turns compared to NH participants.…”

“…When visual cues are present, listeners are less likely to turn one ear to the target speaker in order to increase SNR ( Grange et al, 2018 ). Moreover, it has been shown that while gaze movements accurately follow the target speaker location, head movements tend to undershoot the target location ( Hendrikse et al, 2018 ; Hendrikse et al, 2019a ; Hládek et al, 2019 ; Lu et al, 2021 ). Hendrikse et al (2019a) also found differences between younger and older normal-hearing participants in the relative amount of movement that was done with the head and the eyes (head-eye ratio).…”