DOANet: a deep dilated convolutional neural network approach for search and rescue with drone-embedded sound source localization

Abstract

Drone-embedded sound source localization (SSL) has interesting application perspective in challenging search and rescue scenarios due to bad lighting conditions or occlusions. However, the problem gets complicated by severe drone ego-noise that may result in negative signal-to-noise ratios in the recorded microphone signals. In this paper, we present our work on drone-embedded SSL using recordings from an 8-channel cube-shaped microphone array embedded in an unmanned aerial vehicle (UAV). We use angular spectrum-based TDOA (time difference of arrival) estimation methods such as generalized cross-correlation phase-transform (GCC-PHAT), minimum-variance-distortion-less-response (MVDR) as baseline, which are state-of-the-art techniques for SSL. Though we improve the baseline method by reducing ego-noise using speed correlated harmonics cancellation (SCHC) technique, our main focus is to utilize deep learning techniques to solve this challenging problem. Here, we propose an end-to-end deep learning model, called DOANet, for SSL. DOANet is based on a one-dimensional dilated convolutional neural network that computes the azimuth and elevation angles of the target sound source from the raw audio signal. The advantage of using DOANet is that it does not require any hand-crafted audio features or ego-noise reduction for DOA estimation. We then evaluate the SSL performance using the proposed and baseline methods and find that the DOANet shows promising results compared to both the angular spectrum methods with and without SCHC. To evaluate the different methods, we also introduce a well-known parameter—area under the curve (AUC) of cumulative histogram plots of angular deviations—as a performance indicator which, to our knowledge, has not been used as a performance indicator for this sort of problem before.

Deep Learning Models for Single-Channel Speech Enhancement on Drones

“…The input STFT is calculated using a window length of 2048 and a half overlap. The 10 dilated layers aggregate information across the frequency dimension: they have kernel size (3, 1) and dilation factor 2(d − 1), where d ∈ [1,10] denotes the depth of the layer. The 3 non-dilated layers aggregate information across both time and frequency dimensions, with a kernel size (3,3).…”

Fast Cross-Correlation for TDoA Estimation on Small Aperture Microphone Arrays

“…(22) The bases now hold N/4 + 1 elements (instead of N/2 + 1) and all the elements of p(k) are purely real or imaginary numbers. Computing (22) involves K(N/2 + 2) real multiplications and KN/2 real additions, for a total of K(N +2) flops. Computing the vectors x add (t) and x sub (t) also adds N flops, which leads to a total of K(N + 2) + N flops.…”

“…DoA can also solve the permutation ambiguity in speech separation tasks [12] with multiple microphones, in deep clustering for instance [13], [14], [15] or time-frequency masking [16], [17], [18]. SSL can also serve numerous applications in robotics [19], ranging from acoustic synchronous localization and mapping (SLAM) [20], rescue missions [21], [22], drones tracking [23], [24], [25] and assisting humans with hearing impairments [26]. Some frameworks have been proposed over the years to perform online SSL on robots [27], [28].…”

Deep-Learning-Assisted Sound Source Localization From a Flying Drone

“…DNN sound enhancement is typically a pre-processing step for traditional source localization algorithms [36]. While a DNN can also be trained to predict the location of the sound source directly from the multi-channel microphone signal, the performance typically drops significantly in low-SNR scenarios [37].…”