Aerial Drones with Location-Sensitive Ears

Misra, Prasant; Kumar, Anil; Mohapatra, Pragyan; Balamuralidhar, P.

doi:10.1109/mcom.2018.1700775

Cited by 18 publications

(9 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where C d (•) denotes the contribution of the (k, l)-th bin to the correlation matrix. As shown in (16), this target correlation matrix is crucial to formulate a multi-channel beamformer.…”

Section: Dnn-tfmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning Assisted Time-Frequency Processing for Speech Enhancement on Drones

Wang

Cavallaro

2021

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

This article fills the gap between the growing interest in signal processing based on Deep Neural Networks (DNN) and the new application of enhancing speech captured by microphones on a drone. In this context, the quality of the target sound is degraded significantly by the strong ego-noise from the rotating motors and propellers. We present the first work that integrates single-channel and multi-channel DNN-based approaches for speech enhancement on drones. We employ a DNN to estimate the ideal ratio masks at individual time-frequency bins, which are subsequently used to design three potential speech enhancement systems, namely singlechannel ego-noise reduction (DNN-S), multi-channel beamforming (DNN-BF), and multi-channel time-frequency spatial filtering (DNN-TF). The main novelty lies in the proposed DNN-TF algorithm, which infers the noise-dominance probabilities at individual time-frequency bins from the DNN-estimated soft masks, and then incorporates them into a time-frequency spatial filtering framework for ego-noise reduction. By jointly exploiting the direction of arrival of the target sound, the time-frequency sparsity of the acoustic signals (speech and ego-noise) and the time-frequency noise-dominance probability, DNN-TF can suppress the ego-noise effectively in scenarios with very low signal-to-noise ratios (e.g. SNR lower than −15 dB), especially when the direction of the target sound is close to that of a source of the ego-noise. Experiments with real and simulated data show the advantage of DNN-TF over competing methods, including DNN-S, DNN-BF and the state-of-the-art time-frequency spatial filtering.

show abstract

Section: Dnn-tfmentioning

confidence: 99%

“…3) Spatial Filtering: Given the target correlation matrix, we formulate a standard multi-channel Wiener filter, similarly to (16), as…”

Section: Dnn-tfmentioning

confidence: 99%

Deep Learning Assisted Time-Frequency Processing for Speech Enhancement on Drones

Wang

Cavallaro

2021

IEEE Trans. Emerg. Top. Comput. Intell.

View full text Add to dashboard Cite

show abstract

“…9 These auditory systems have been used in conjunction with radars and acoustic sensors, showing good performances when detecting UAVs in public places 10 and detecting sound sources in spaces of interest. 11 Even though these alternatives have been developed, the audio processing area of research over a UAV is a challenging task that still has considerable room to develop.…”

Section: Related Workmentioning

confidence: 99%

Detection of nearby UAVs using a multi-microphone array on board a UAV

Cabrera-Ponce

Martínez-Carranza

Rascón

2020

International Journal of Micro Air Vehicles

View full text Add to dashboard Cite

In this work, we address the problem of UAV detection flying nearby another UAV. Usually, computer vision could be used to face this problem by placing cameras onboard the patrolling UAV. However, visual processing is prone to false positives, sensible to light conditions and potentially slow if the image resolution is high. Thus, we propose to carry out the detection by using an array of microphones mounted with a special array onboard the patrolling UAV. To achieve our goal, we convert audio signals into spectrograms and used them in combination with a CNN architecture that has been trained to learn when a UAV is flying nearby, and when it is not. Clearly, the first challenge is the presence of ego-noise derived from the patrolling UAV itself through its propellers and motor’s noise. Our proposed CNN is based on Google’s Inception v.3 network. The Inception model is trained with a dataset created by us, which includes examples of when an intruder UAV flies nearby and when it does not. We conducted experiments for off-line and on-line detection. For the latter, we manage to generate spectrograms from the audio stream and process it with the Nvidia Jetson TX2 mounted onboard the patrolling UAV.

show abstract

“…In contrast, for works falling into the LA and AA categories, UAVs have to be adapted to carry out the microphone arrays; hence, custom UAVs may be more convenient. [59] AR drone [60] 3DR IRIS [61] Quad [24] Hexa [24] Hexa [62] Ballon [23] Bebop [23] Pelican [23] Zion [23] Quad [63] 2018 * Matrice 600 pro [47] AR drone [64] Fixed-Wing [65] Bebop [66] Quad [21] Quad [20] Quad [67] Hexa [68] 2019 Fixed-Wing [69] Quad [70] Quad [71] Bebop [72] Bebop [14] Bebop 2 [15] Matrice 100 [15] MK-Quadro [73]…”

Section: On the Number Of Microphones And Uav Modelsmentioning

confidence: 99%

A Review on Auditory Perception for Unmanned Aerial Vehicles

Martínez-Carranza

Rascón

2020

Sensors

View full text Add to dashboard Cite

Although a significant amount of work has been carried out for visual perception in the context of unmanned aerial vehicles (UAVs), not so much has been done regarding auditory perception. The latter can complement the observation of the environment that surrounds a UAV by providing additional information that can be used to detect, classify, and localize audio sources of interest. Motivated by the usefulness of auditory perception for UAVs, we present a literature review that discusses the audio techniques and microphone configurations reported in the literature. A categorization of techniques is proposed based on the role a UAV plays in the auditory perception (is it the one being perceived or is it the perceiver?), as well as a set of objectives that are more popularly aimed to be accomplished in the current literature (detection, classification, and localization). This literature review aims to provide a concise landscape of the most relevant works on auditory perception in the context of UAVs to date and provides insights into future avenues of research as a guide to those who are beginning to work in this field.

show abstract

Aerial Drones with Location-Sensitive Ears

Cited by 18 publications

References 13 publications

Deep Learning Assisted Time-Frequency Processing for Speech Enhancement on Drones

Deep Learning Assisted Time-Frequency Processing for Speech Enhancement on Drones

Detection of nearby UAVs using a multi-microphone array on board a UAV

A Review on Auditory Perception for Unmanned Aerial Vehicles

Contact Info

Product

Resources

About