Onboard Powerline Perception System for UAVs Using mmWave Radar and FPGA-Accelerated Vision

Abstract

Autonomous Unmanned Aerial Vehicle (UAV) interactions with powerlines, such as close-up inspections for fault detection or grasping and landing for recharging, require advanced onboard perception capabilities. To solve such tasks, the UAV must be equipped with perception abilities that allow it to navigate between powerlines and safely approach specific cables of interest. A perception system with such capabilities requires state-of-the-art sensor technologies and data processing while still being subject to the limited hardware and energy resources of the UAV. In this paper, we present an advanced embedded system based on the cutting-edge Multiprocessing System-on-Chip (MPSoC) for onboard UAV powerline perception. Our platform consists of a mmWave radar and an RGB camera with data processing carried out on the MPSoC, covering both CPU and Field-Programmable Gate Array (FPGA) computations. Following hardware-software co-design methodology, the heavy image processing tasks are accelerated in the FPGA and fused with computationally light mmWave data on the CPU, facilitating pose-estimation of the power lines. Utilizing the open-source autonomy frameworks PX4 and ROS2, we demonstrate integration of the system with onboard path planning based on the estimated cable positions. The robustness of the detection and pose-estimation methods have been demonstrated in several tests performed both in simulated and real-world powerline environments. The results show that our proposed perception system allows the UAV to safely navigate in close proximity to powerlines, by perceiving more individual cables at longer distances compared to previous work, while remaining lightweight, power-efficient, and low-cost.

Design of an Autonomous Cooperative Drone Swarm for Inspections of Safety Critical Infrastructure

“…Figure 6 shows the test environment and the data generated by the perception system. A more in-depth explanation of this work can be found in [44].…”

Análisis de un control crucero adaptable en condiciones de poca visibilidad

“…Dado que un FPGA puede otorgar prestaciones según las necesidades específicas del sistema. Se considera que usar un FPGA daría lugar a un sistema superior de desarrollo en comparación con las alternativas mencionadas anteriormente, véase (Malle, Nyboe, & Ebeid, 2022).…”

Open-Source Hardware/Software Architecture for Autonomous Powerline-Aware Drone Navigation and Recharging

“…A conceptual diagram of the full computational system is seen in Fig. 2 board runs the perception system as presented in [13], and another Ultra96-V2 runs the navigation software. The sensor package, consisting of a mmWave radar sensor and an RGB camera, is connected to the perception Ultra96-V2 through USB.…”

“…The perception system running on the perception Ultra96-V2 was previously presented by the authors in [13]. It fuses images from a USB camera with radar data from a Texas Instruments IWR6843AOP [14].…”