Reverberant sound localization with a robot head based on direct-path relative transfer function

Abstract

This paper addresses the problem of sound-source localization (SSL) with a robot head, which remains a challenge in real-world environments. In particular we are interested in locating speech sources, as they are of high interest for humanrobot interaction. The microphone-pair response corresponding to the direct-path sound propagation is a function of the source direction. In practice, this response is contaminated by noise and reverberations. The direct-path relative transfer function (DP-RTF) is defined as the ratio between the directpath acoustic transfer function (ATF) of the two microphones, and it is an important feature for SSL. We propose a method to estimate the DP-RTF from noisy and reverberant signals in the short-time Fourier transform (STFT) domain. First, the convolutive transfer function (CTF) approximation is adopted to accurately represent the impulse response of the microphone array, and the first coefficient of the CTF is mainly composed of the direct-path ATF. At each frequency, the frame-wise speech auto-and cross-power spectral density (PSD) are obtained by spectral subtraction. Then a set of linear equations is constructed by the speech auto-and cross-PSD of multiple frames, in which the DP-RTF is an unknown variable, and is estimated by solving the equations. Finally, the estimated DP-RTFs are concatenated across frequencies and used as a feature vector for SSL. Experiments with a robot, placed in various reverberant environments, show that the proposed method outperforms two state-of-the-art methods.

Online Localization and Tracking of Multiple Moving Speakers in Reverberant Environments

“…at the beginning of Speaker 3's trajectory (in blue). The possible reasons are i) the NAO robot (v5) has a relative strong egonoise [38], and thus the signal-to-noise ratio of the recorded signals is relative low, and ii) the speakers are moving with a varying source-to-robot distance and the direct-path speech is contaminated by more reverberations when the speakers are distant. Overall, DPRTF-REM and DPRTF-EG are able to monitor the moving, appearance, and disappearance of active speakers for most of the time, with a small time lag due to the temporal smoothing.…”

AcousticFusion: Fusing Sound Source Localization to Visual SLAM in Dynamic Environments

“…Hospedales et al [16] proposed a Bayesian model-based audio-visual fusion framework to segment, associate, and track multiple objects in audiovisual sequences. Li et al presented an SSL-based HRI system in [17]. They calibrated the sound sources' corresponding pixel coordinates.…”

“…The audio-visual fusion works [17], [18], [19], all use static robots to track the observer. For the moving robot, in [20], Evers et al proposed an acoustic SLAM framework that is different from the general concept of SLAM.…”

Online Localization of Multiple Moving Speakers in Reverberant Environments

“…Experiments with real data are conducted using a version 5 NAO robot whose head has four microphones in a horizontal plane [22]. Thence we only perform 360 k is computed using the HRTF of NAO.…”