No abstract

Unmanned Aerial Vehicles (UAVs) are promising technologies within many different application scenarios including human detection in search and rescue and surveillance use cases, which have received considerable attention worldwide. However, adverse conditions, such as varying altitude, overhead camera placement, changing illumination and moving platform, impose challenges for highperformance yet cost-efficient human detection. To overcome these challenges, we propose a novel combination of dilated convolutions with Path Aggregation Network (PAN) as a new deep neural network-based human detection algorithm in real time. Furthermore, we establish a comprehensive human detection dataset with varying backgrounds, illuminations, and contrast and train the proposed machine-learning model on the collected dataset. Our approach achieves both high precision (88.0% mean Average Precision (mAP)) and real time (67.0 Frames Per Second (FPS)) on a commercial off-the-shelf PC platform. In terms of accuracy, the result is comparable to the standard You Only Look Once v3 (YOLOv3). However, the speed is twice as that of the standard YOLOv3. YOLOv4 is slightly more accurate (89.8%) than our approach. However, it is slower (38.0 versus 67.0 FPS) and has more Billion Floating-Point Operations (BFLOPS). The proposed algorithm has also trained with the VisDrone2019 dataset and compared with seven studies using this dataset. The results have further validated the effectiveness of the proposed approach. Moreover, the algorithm has been evaluated on an embedded system (Jetson AGX Xavier), which demonstrates the usefulness of this method on power-constrained devices. The proposed algorithm is fast, memory efficient, and computationally less expensive to achieve high detection performance. It is expected to contribute significantly to the wider use of UAV applications including search and rescue missions to locate missing people, and surveillance particularly for applications running on resource-constrained platforms, like smartphones or tablets. This proposed system is now being used in aerial drone system of Police of Scotland to help them locate and find missing and vulnerable people. The results of the project were broadcasted by BBC Scotland.

Object detection systems mounted on Unmanned Aerial Vehicles (UAVs) have gained momentum in recent years in light of the widespread use cases enabled by such systems in public safety and other areas. Machine learning has emerged as an enabler for improving the performance of object detection. However, there is little existing work that has studied the performance of the machine learning approach, which is computationally resource demanding, in a portable mobile platform for UAV based object detection in user mobility scenarios. This paper evaluates an integrated real-world testbed for this scenario, by employing commercial-off-the-shelf devices including a UAV system and a machine-learning-enabled mobile platform. It presents benchmarking results about the performance of popular machine learning and computer vision frameworks such as TensorFlow and OpenCV and the associated algorithms such as YOLO, embedded in a smartphone execution environment of limited resources. The results highlight opportunities and provide insights into technical gaps to be filled to realize real-time machine-learning-based object detection on a mobile platform with constrained resources.

Existing artificial intelligence solutions typically operate in powerful platforms with high computational resources availability. However, a growing number of emerging use cases such as those based on unmanned aerial systems (UAS) require new solutions with embedded artificial intelligence on a highly mobile platform. This paper proposes an innovative UAS that explores machine learning (ML) capabilities in a smartphone‐based mobile platform for object detection and recognition applications. A new system framework tailored to this challenging use case is designed with a customized workflow specified. Furthermore, the design of the embedded ML leverages TensorFlow, a cutting‐edge open‐source ML framework. The prototype of the system integrates all the architectural components in a fully functional system, and it is suitable for real‐world operational environments such as seek and rescue use cases. Experimental results validate the design and prototyping of the system and demonstrate an overall improved performance compared with the state of the art in terms of a wide range of metrics.

This paper proposes an acceleration technique to minimise the unnecessary operations on a state-of-the-art machine learning model and thus to improve the processing speed while maintaining the accuracy. After the study of the main bottlenecks that negatively affect the performance of convolutional neural networks, this paper designs and implements a discarding technique for YOLOv3-based algorithms to increase the speed and maintain accuracy. After applying the discarding technique, YOLOv3 can achieve a 22% of improvement in terms of speed. Moreover, the results of this new discarding technique were tested on Tiny-YOLOv3 with three output layers on an autonomous vehicle for pedestrian detection and it achieved an improvement of 48.7% in speed. The dynamic discarding technique just needs one training process to create the model and thus execute the approach, which preserves accuracy. The improved detector based on the discarding technique is able to readily alert the operator of the autonomous vehicle to take the emergency brake of the vehicle in order to avoid collision and consequently save lives.

Machine learning algorithms based on convolutional neural networks (CNNs) have recently been explored in a myriad of object detection applications. Nonetheless, many devices with limited computation resources and strict power consumption constraints are not suitable to run such algorithms designed for high-performance computers. Hence, a novel smartphone-based architecture intended for portable and constrained systems is designed and implemented to run CNN-based object recognition in real time and with high efficiency. The system is designed and optimised by leveraging the integration of the best of its kind from the state-of-the-art machine learning platforms including OpenCV, TensorFlow Lite, and Qualcomm Snapdragon informed by empirical testing and evaluation of each candidate framework in a comparable scenario with a high demanding neural network. The final system has been prototyped combining the strengths from these frameworks and led to a new machine learning-based object recognition execution environment embedded in a smartphone with advantageous performance compared with the previous frameworks.

cost-effective object recognition in an integrated smartphone and UAV system. In 2020 European Conference on Networks and Communications (EuCNC) (pp. 316-320). (IEEE Conference Proceedings). IEEE.

To improve the speed and accuracy in human detection in Search and Rescue (SAR) operations, this paper presents a novel and highly efficient machine learning empowered system by extending the You Only Look Once (YOLO) algorithm, which is designed and deployed on an embedded system. The proposed approach has been evaluated under real-world conditions on a Jetson AGX Xavier platform and the results have shown a well-balanced system in terms of accuracy, speed and portability. Moreover, the system demonstrates its resilience to perform low-pixel human detection on infrared images received from an Unmanned Aerial Vehicle (UAV) at low-light conditions, different altitudes and postures such as sitting, walking and running. The proposed approach has achieved in a constrained environment a total of 89.26% of accuracy and 24.6 FPS, surpassing the barrier of real-time object recognition. CCS CONCEPTS• Computing methodologies → Neural networks.