Deep Learning on Multi Sensor Data for Counter UAV Applications—A Systematic Review

Abstract

Usage of Unmanned Aerial Vehicles (UAVs) is growing rapidly in a wide range of consumer applications, as they prove to be both autonomous and flexible in a variety of environments and tasks. However, this versatility and ease of use also brings a rapid evolution of threats by malicious actors that can use UAVs for criminal activities, converting them to passive or active threats. The need to protect critical infrastructures and important events from such threats has brought advances in counter UAV (c-UAV) applications. Nowadays, c-UAV applications offer systems that comprise a multi-sensory arsenal often including electro-optical, thermal, acoustic, radar and radio frequency sensors, whose information can be fused to increase the confidence of threat’s identification. Nevertheless, real-time surveillance is a cumbersome process, but it is absolutely essential to detect promptly the occurrence of adverse events or conditions. To that end, many challenging tasks arise such as object detection, classification, multi-object tracking and multi-sensor information fusion. In recent years, researchers have utilized deep learning based methodologies to tackle these tasks for generic objects and made noteworthy progress, yet applying deep learning for UAV detection and classification is considered a novel concept. Therefore, the need to present a complete overview of deep learning technologies applied to c-UAV related tasks on multi-sensor data has emerged. The aim of this paper is to describe deep learning advances on c-UAV related tasks when applied to data originating from many different sensors as well as multi-sensor information fusion. This survey may help in making recommendations and improvements of c-UAV applications for the future.

Defending Airports from UAS: A Survey on Cyber-Attacks and Counter-Drone Sensing Technologies

“…Several studies analyzed a monostatic radar working either at 35 GHz [ 41 ] or at 9.4 GHz [ 42 ] to detect and track nearby drones. The most employed radar signal characteristic for automatic target classification is the micro-Doppler (m-D) signature [ 43 , 44 ]. The intrinsic rotation movements of UAV rotor blades can define the type of drone, while the propulsion turbine of a jet or the flapping wings of a bird can be statistically described by the radar m-D signature [ 44 , 45 , 46 ].…”

Classification of VLF/LF Lightning Signals Using Sensors and Deep Learning Methods

“…Deep learning [16] is a rapidly growing area of machine learning, which has been widely used in the fields of image recognition, speech analysis, and medical diagnosis in recent years [17][18][19][20]. Recently, Zaiwar Ali et al [21] proposed an energy-saving deep learning shunting scheme to train a deep learning-based intelligent decision-making algorithm; Mohamed Alloghani et al [22] applied machine learning algorithms to the clustering and prediction of vital signs; Stamatios Samaras et al [23] described the progress of deep learning of multi-sensor information fusion in drone applications. Heena Rathore et al [24] proposed a new deep learning strategy for classifying different attack methods of deep brain implants.…”

A Machine Learning Method for Vision-Based Unmanned Aerial Vehicle Systems to Understand Unknown Environments

“…Hence, RGB images in this paper represent the frames of video captured by the UAV. The UAVs are always equipped with multiple integrated sensors for understanding, mainly through radar and/or electro-optical/thermal (EO-IR) sensors and less commonly through acoustic and radio frequency (RF) sensors [12]. Compared with other types electro-optical sensors (like cameras) are cheap, sensitive to environmental settings, and can detect and classify with the highest capability if the target is visible.…”