MPDrone: FPGA-based Platform for Intelligent Real-time Autonomous Drone Operations

Kovari, Balint Barna; Ebeid, Emad

doi:10.1109/ssrr53300.2021.9597857

Cited by 9 publications

(3 citation statements)

References 12 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 2020, Moreac et al [23] proposed FPGA based computer vision for drones by hardware-in-the-loop simulation. Finally in 2021 at our research group, Kövari [24] demonstrated the use of FPGA accelerated deep learning inference for autonomous drone navigation, also based on PYNQ.…”

Section: Related Workmentioning

confidence: 99%

MPSoC4Drones: An Open Framework for ROS2, PX4, and FPGA Integration

Nyboe

Malle

Ebeid

2022

2022 International Conference on Unmanned Aircraft Systems (ICUAS)

Self Cite

View full text Add to dashboard Cite

Autonomous drones are facing a tough efficiency challenge due to limitations on the utilized processing hardware units. Among these limitations is the tradeoff between fast computing and low power consumption; between functional complexity and flight time. Recent progressions point to FPGAs for accelerating heavy processing. In this work, we present the MPSoC4Drones Framework; a novel framework for organizing FPGA-design and OS build projects. The framework combines tools for creating bootable images for the Ultra96-V2 board.We show how MPSoC4Drones organizes the build, combining the latest well-known tools for research and industrial drone development, Ubuntu 20.04, PX4 autopilot, and ROS2 middleware. We validate the framework through a computationally intensive deep learning use case implemented in the MPSoC4Drones framework. We show the superior throughput and low power consumption of FPGA processing compared to classical CPU and GPU approaches. Finally, we offer the full framework as open-source.

show abstract

Section: Related Workmentioning

confidence: 99%

MPSoC4Drones: An Open Framework for ROS2, PX4, and FPGA Integration

Nyboe

Malle

Ebeid

2022

2022 International Conference on Unmanned Aircraft Systems (ICUAS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many IoT networks require high amounts of data throughput, especially those that integrate high resolution video feeds such as applications involving drones [ 53 , 54 , 55 , 56 ]. FPGA designers have addressed these challenge of increasing networking throughput by implementing FPGAs as Network Interface Cards (NICs); it is known that FPGA’s capabilities for parallel processing can be utilized to create entire TCP/IP stacks.…”

Section: High Performance Networkmentioning

confidence: 99%

Review of State-of-the-Art FPGA Applications in IoT Networks

Magyari

Chen

2022

Sensors

View full text Add to dashboard Cite

Modern networks used for integrating custom Internet of Things (IoT) systems and devices have restrictions and requirements unique to their individual applications. These application specific demands require custom designed hardware to maximize throughput, security and data integrity whilst minimizing latency, power consumption, and form factor. Within this paper, we describe current, state-of-the-art works that utilize FPGAs for IoT network developments. We analyze two categories of works: those that prioritize reducing power consumption, and those that prioritize networking features. Further, we describe how future works can improve upon these designs and therefore improve the efficiency of resource-constrained IoT networks.

show abstract

“…Additionally, none of the methods have been demonstrated to run onboard a UAV in real-time. Because of their power efficiency and high throughput capability, FPGAs have been used on UAVs as flight controllers [40], neural network accelerators [41], [42], and generally deployed for their reconfigurability [43], [44]. NASA's Mars UAV [45] uses FPGAs for flight control, fault tolerance, and as IO and communications hubs.…”

Section: Related Workmentioning

confidence: 99%

Onboard Powerline Perception System for UAVs Using mmWave Radar and FPGA-Accelerated Vision

2022

View full text Add to dashboard Cite

Autonomous Unmanned Aerial Vehicle (UAV) interactions with powerlines, such as close-up inspections for fault detection or grasping and landing for recharging, require advanced onboard perception capabilities. To solve such tasks, the UAV must be equipped with perception abilities that allow it to navigate between powerlines and safely approach specific cables of interest. A perception system with such capabilities requires state-of-the-art sensor technologies and data processing while still being subject to the limited hardware and energy resources of the UAV. In this paper, we present an advanced embedded system based on the cutting-edge Multiprocessing System-on-Chip (MPSoC) for onboard UAV powerline perception. Our platform consists of a mmWave radar and an RGB camera with data processing carried out on the MPSoC, covering both CPU and Field-Programmable Gate Array (FPGA) computations. Following hardware-software co-design methodology, the heavy image processing tasks are accelerated in the FPGA and fused with computationally light mmWave data on the CPU, facilitating pose-estimation of the power lines. Utilizing the open-source autonomy frameworks PX4 and ROS2, we demonstrate integration of the system with onboard path planning based on the estimated cable positions. The robustness of the detection and pose-estimation methods have been demonstrated in several tests performed both in simulated and real-world powerline environments. The results show that our proposed perception system allows the UAV to safely navigate in close proximity to powerlines, by perceiving more individual cables at longer distances compared to previous work, while remaining lightweight, power-efficient, and low-cost.

show abstract

MPDrone: FPGA-based Platform for Intelligent Real-time Autonomous Drone Operations

Cited by 9 publications

References 12 publications

MPSoC4Drones: An Open Framework for ROS2, PX4, and FPGA Integration

MPSoC4Drones: An Open Framework for ROS2, PX4, and FPGA Integration

Review of State-of-the-Art FPGA Applications in IoT Networks

Onboard Powerline Perception System for UAVs Using mmWave Radar and FPGA-Accelerated Vision

Contact Info

Product

Resources

About