Detection of activity and position of speakers by using deep neural networks and acoustic data augmentation

Abstract

The task of Speaker LOCalization (SLOC) has been the focus of numerous works in the research field, where SLOC is performed on pure speech data, requiring the presence of an Oracle Voice Activity Detection (VAD) algorithm. Nevertheless, this perfect working condition is not satisfied in a real world scenario, where employed VADs do commit errors. This work addresses this issue with an extensive analysis focusing on the relationship between several datadriven VAD and SLOC models, finally proposing a reliable framework for VAD and SLOC. The effectiveness of the approach here discussed is assessed against a multi-room scenario, which is close to a real world environment. Furthermore, up to the authors' best knowledge, only one contribution proposes a unique framework for VAD and SLOC acting in this addressed scenario; however this solution does not rely on data-driven approaches. This work comes as an extension of the authors' previous research addressing the VAD and SLOC tasks, by proposing numerous advancements to the original neural network architectures. In details, four different models based on convolutional neural networks (CNNs) are here tested, in order to easily highlight the advantages of the introduced novelties. In addition, two different CNN models go under study for SLOC. Furthermore, training of data-driven models is here improved through a specific data augmentation technique. During this procedure, the room impulse responses (RIRs) of two virtual rooms are generated from the knowledge of the room size, reverberation time and microphones and sources placement. Finally, the only other framework for simultaneous detection and localization in a multi-room scenario is here taken into account to fairly compare the proposed method. As result, the proposed method shows to be more accurate than the baseline framework, and remarkable improvements are specially observed when the data

Overlapped Speech Detection and speaker counting using distant microphone arrays

“…In addition, we explore the use of spatial features to aid VAD+OSD and speaker counting. As mentioned above, a number of works have shown that spatial features can be used for counting (Drude et al, 2014;Pasha et al, 2017;Brutti et al, 2010;Pavlidi et al, 2012) and VAD (Vecchiotti et al, 2019b). However, to our knowledge, no study has yet been performed where spatial features are used in conjunction with deep neural networks to tackle OSD and speaker counting directly.…”

A Survey of Machine Learning in Pedestrian Localization Systems: Applications, Open Issues and Challenges

“…Both techniques are widely used thanks to their high accuracy and relatively low processing cost. For instance, the authors in [24], [26], [27], [44], [49], and [51] use CNN and RNN to estimate the pedestrian localization, improving the fingerprint creation process, using different data-training, and reducing the noise effects. The best result shows an improvement on accuracy by 75% when compared to the pedestrian localization system without the use of ML.…”

Microphone Array Based Surveillance Audio Classification

“…In recent years, researchers have shown that the most effective tools for the classification of sound events include the application of deep, convolutional, and recurrent neural networks (DNN, CNN, and RNN) [7], [8], [3], [9], [4]. However, for the current work, the concern with the processing time of the algorithms is fundamental, since, among the future goals, the aim is to create a low-cost system capable of running in real-time.…”

“…Recent work advocates the use of DNN and CNN can perceive patterns in auditors without using many features [9], [7]. Both were able to acquire good results using only Mel Frequency Cepstral Coefficients (MFCC).…”

“…This causes directivity patterns to be created, amplifying signals from a certain direction, and attenuating from others [15], [16]. To find the lags of each microphone concerning the referential, auto-correlation analysis in the time or frequency domain can be performed [17], [7]. The so-called Generalized Sidelobe Canceller differs from the previous one for being adaptive.…”