Jointly optimal near-end and far-end multi-microphone speech intelligibility enhancement based on mutual information

Abstract

The processing required for the global maximization of the intelligibility of speech acquired by multiple microphones and rendered by a single loudspeaker, is considered in this paper. The intelligibility is quantized, based on the mutual information rate between the message spoken by the talker and the message as interpreted by the listener. We prove that then, in each of a set of narrow-band channels, the processing can be decomposed into a minimum variance distortionless response (MVDR) beamforming operation that reduces the noise in the talker environment, followed by a gain operation that, given the far-end noise and beamforming operation, accounts for the noise at the listener end. Our experiments confirm that both processing steps are necessary for the effective conveyance of a message and, importantly, that the second step must be aware of the first step.Index Terms-Speech intelligibility enhancement, mutual information, minimum variance distortionless response (MVDR) beamformer, multi-microphone.

Joint Far- and Near-End Speech Intelligibility Enhancement Based on the Approximated Speech Intelligibility Index

“…MI-based methods for joint far-and near-end SI enhancement [8,10] improve SI by maximizing the MI, I(S; Z), between the clean speech, S, and the signal received by the listener, Z.…”

“…In [9] a new training strategy is proposed for deep learning based single-channel enhancement given that speech has already been processed at the far-end. For joint multi-microphone FSE and NLE [8,10] proposes to optimize the Mutual Information (MI) [11] between the clean speech and the signal received by the listener. The results of [8] are the first to show both theoretically and experimentally that joint processing, using knowledge of processing and conditions at both ends, is superior to the classic disjoint processing.…”