Vision-Based Mid-Air Object Detection and Avoidance Approach for Small Unmanned Aerial Vehicles with Deep Learning and Risk Assessment

Abstract

With the increasing demand for unmanned aerial vehicles (UAVs), the number of UAVs in the airspace and the risk of mid-air collisions caused by UAVs are increasing. Therefore, detect and avoid (DAA) technology for UAVs has become a crucial element for mid-air collision avoidance. This study presents a collision avoidance approach for UAVs equipped with a monocular camera to detect small fixed-wing intruders. The proposed system can detect any size of UAV over a long range. The development process consists of three phases: long-distance object detection, object region estimation, and collision risk assessment and collision avoidance. For long-distance object detection, an optical flow-based background subtraction method is utilized to detect an intruder far away from the host. A mask region-based convolutional neural network (Mask R-CNN) model is trained to estimate the region of the intruder in the image. Finally, the collision risk assessment adopts the area expansion rate and bearing angle of the intruder in the images to conduct mid-air collision avoidance based on visual flight rules (VFRs) and conflict areas. The proposed collision avoidance approach is verified by both simulations and experiments. The results show that the system can successfully detect different sizes of fixed-wing intruders, estimate their regions, and assess the risk of collision at least 10 s in advance before the expected collision would happen.

Research on Bidirectional Multi-Span Feature Pyramid and Key Feature Capture Object Detection Network

“…In article [28], a joint motion mechanism based on a three-degree-of-freedom (DOF) framework was designed for drones in complex motion patterns to achieve real-time active tracking of targets. Lai et al [29] proposed a background subtraction method to detect moving targets and used the Mask R-CNN model to identify target types. Article [30] provides an overview of the 2022 L4S competition aimed at overcoming the challenges of detecting landslides in remote sensing images.…”

Optical Flow-Based Obstacle Detection for Mid-Air Collision Avoidance

“…The utilization of traditional computer vision techniques presents a significant advantage over deep neural networks due to the latter's reliance on extensive training datasets. Such datasets, particularly those concerning mid-air collisions, are exceptionally challenging to acquire, rendering traditional methods more feasible and effective in this context [12].…”