On Improved Training of CNN for Acoustic Source Localisation

Abstract

Convolutional Neural Networks (CNNs) are a popular choice for estimating Direction of Arrival (DoA) without explicitly estimating delays between multiple microphones. The CNN method first optimises unknown filter weights (of a CNN) by using observations and ground-truth directional information. This trained CNN is then used to predict incident directions given test observations. Most existing methods train using spectrallyflat random signals and test using speech. In this paper, which focuses on single source DoA estimation, we find that training with speech or music signals produces a relative improvement in DoA accuracy for a variety of audio classes across 16 acoustic conditions and 9 DoAs, amounting to an average improvement of around 17% and 19% respectively when compared to training with spectrally flat random signals. This improvement is also observed in scenarios in which the speech and music signals are synthesised using, for example, a Generative Adversarial Network (GAN). When the acoustic environments during test and training are similar and reverberant, training a CNN with speech outperforms Generalized Cross Correlation (GCC) methods by about 125%. When the test conditions are different, a CNN performs comparably. This paper takes a step towards answering open questions in the literature regarding the nature of the signals used during training, as well as the amount of data required for estimating DoA using CNNs.

Improvement of learning-based methods for localization of multiple sound sources

“…Multiple investigations of application of ANNs for SSL were presented recently (Argentieri et al 2015;Grumiaux et al 2021b). Sound source localization using ANN is commonly formulated either as a classification (Bohlender et al 2021;Chakrabarty, Habets 2019a;Grumiaux et al 2021a;Hao et al 2020;Hirvonen 2015;Hubner et al 2021;Ma, Liu 2018;Roden et al 2015;Vargas et al 2021), or a regression problem (Cao et al 2019;Grondin et al 2019;Kim 2014;Pertila, Cakir 2017;Youssef et al 2013).…”

“…While there are many available ANN types, most commonly, the CNN (Salvati et al 2018;Vargas et al 2021) and RNN (Wang et al 2019) or a combination of both are used (Huang et al 2018;Ma et al 2015;Roden et al 2015;Takeda, Komatani 2016a;Youssef et al 2013) and Autoencoders (Huang et al 2020;Wu et al 2021;Zermini et al 2016) are also investigated.…”

Estimation of Azimuth and Elevation for Multiple Acoustic Sources Using Tetrahedral Microphone Arrays and Convolutional Neural Networks

“…An application of recurrent neural networks (RNNs) for SSL was investigated in [19]. Sound source localization using ANN is commonly formulated either as a classification [20][21][22][23][24][25][26][27][28], or a regression problem [29][30][31][32][33]. In the case of the regression problem, the output of the ANN is a one, two or three-dimensional vector (in the case of a single sound source [34,35]) or a set of vectors (in the case of a multiple source localization [36][37][38]).…”

Block-Based Compression and Corresponding Hardware Circuits for Sparse Activations

“…Nowadays, convolutional neural networks (CNNs) [1,2] have been widely used in many application fields, such as computer vision [3,4], signal processing [5,6] and image processing [7,8]. Note that a CNN is composed of multiple layers.…”