Efficient Real-Time Human Detection Using Unmanned Aerial Vehicles Optical Imagery

Abstract

Unmanned Aerial Vehicles (UAVs) are promising technologies within many different application scenarios including human detection in search and rescue and surveillance use cases, which have received considerable attention worldwide. However, adverse conditions, such as varying altitude, overhead camera placement, changing illumination and moving platform, impose challenges for highperformance yet cost-efficient human detection. To overcome these challenges, we propose a novel combination of dilated convolutions with Path Aggregation Network (PAN) as a new deep neural network-based human detection algorithm in real time. Furthermore, we establish a comprehensive human detection dataset with varying backgrounds, illuminations, and contrast and train the proposed machine-learning model on the collected dataset. Our approach achieves both high precision (88.0% mean Average Precision (mAP)) and real time (67.0 Frames Per Second (FPS)) on a commercial off-the-shelf PC platform. In terms of accuracy, the result is comparable to the standard You Only Look Once v3 (YOLOv3). However, the speed is twice as that of the standard YOLOv3. YOLOv4 is slightly more accurate (89.8%) than our approach. However, it is slower (38.0 versus 67.0 FPS) and has more Billion Floating-Point Operations (BFLOPS). The proposed algorithm has also trained with the VisDrone2019 dataset and compared with seven studies using this dataset. The results have further validated the effectiveness of the proposed approach. Moreover, the algorithm has been evaluated on an embedded system (Jetson AGX Xavier), which demonstrates the usefulness of this method on power-constrained devices. The proposed algorithm is fast, memory efficient, and computationally less expensive to achieve high detection performance. It is expected to contribute significantly to the wider use of UAV applications including search and rescue missions to locate missing people, and surveillance particularly for applications running on resource-constrained platforms, like smartphones or tablets. This proposed system is now being used in aerial drone system of Police of Scotland to help them locate and find missing and vulnerable people. The results of the project were broadcasted by BBC Scotland.

Smartphone-based real-time object recognition architecture for portable and constrained systems

“…Several approaches could be taken to further reduce the inference time [19], such as quantisation techniques or faster feature Fig. 11 Comparison of our approach with a ideal model that achieves real-time recognition per frame extraction methods [10]. Nevertheless, they may reduce the accuracy.…”

Illumination-aware image fusion for around-the-clock human detection in adverse environments from Unmanned Aerial Vehicle

“…To improve the accuracy of small object detection, additional techniques are needed. Inspired by the Feature Pyramid Network (FPN) (Lin et al, 2017), the accuracy of small objects is improved by using the PAN architecture (Golcarenarenji et al, 2021(Golcarenarenji et al, , 2022Liu et al, 2018b). This is a method that improves the accuracy of small object detection by reducing the data path between the initial layers and deeper layers.…”

“…To achieve high-accuracy, high-speed, and portability which is imperative for the success of primary Search and rescue missions used by police of Scotland and other applicable use cases such as intruder detection from UAVs in the EU Horizon 2020 5G-PPP 5G-INDUCE project and Angel Drone in EU Horizon 2020 ARCADIAN-IoT project, speed of FPS ≥24, accuracy of more than 90% (to find missing people and save lives), and resource efficiency for portability of the solutions on constrained devices such as smartphone, and tablets (Model size≤ 54 MB and BFLOPS ≤ 28) are required (Golcarenarenji et al, 2021;Martinez-Alpiste et al, 2020c;Martinez-Alpiste et al, 2019;Martinez-Alpiste et al, 2020a). Hence, the following steps are proposed solutions to achieve the requirements of these projects.…”

Machine-learning-based top-view safety monitoring of ground workforce on complex industrial sites

“…Inspired by the Feature Pyramid Network (FPN) [24], PAN [25,26] is a method that improves the accuracy of small object detection by adding an additional bottom-up path augmentation to combine features from initial layers with more detailed information and the deeper layers with more meaningful information as both information is needed to improve small object detection. In other words, the data path between lower layers and top layer is reduced using the PAN architecture.…”

“…However, the max-pooling strides were selected as 4, 8 and 16 different from those of [44]. A PAN (Path Aggregation Network) module [25,26] was also integrated (labelled as c) in the algorithm which consists of a top-down pathway with Fig. 3 The architecture of the proposed algorithm.…”