An Optimization Framework for Efficient Vision-Based Autonomous Drone Navigation

Abstract

Fully autonomous drones are a new emerging field that has enabled many applications such as gas source leakage localization, wild-fire detection, smart agriculture, and search and rescue missions in unknown limited communication and GPS denied environments. Artificial intelligence and deep Neural Networks (NN) have enabled applications such as visual perception and navigation which can be deployed to make drones smarter and more efficient. However, deploying such techniques on tiny drones is extremely challenging due to the limited computational resources and power envelope of edge devices. To achieve this goal, this paper proposes an efficient end-to-end optimization method for deploying deep NN models for visionbased autonomous drone navigation applications, such as obstacle avoidance and steering task. This paper formulates two different methods for implementing the NN inference phase onto tiny drones and analyzing the implementation results for each case: 1) a Cloud-IoT implementation and 2) Onboard Processing. Several models are trained with state-of-the-art scalable NN architectures and the most efficient cases in terms of computation complexity and accuracy are selected for implementation on a cloud server and several edge devices. By designing hardware-friendly NN models and optimal configuration of the implementation platforms, we were able to reach up to 97% accuracy, speed up the computation 2.3x, have 22x less complexity, and 53% energy reduction. Also, we achieve up to 25 fps on the GAP8 processor, which is enough for real-time drone navigation requirements, even when the model is running on a small IoT device.

MLAE2: Metareasoning for Latency-Aware Energy-Efficient Autonomous Nano-Drones

“…The main application that we considered in this paper is autonomous drone navigation. We used two energy-efficient pre-trained models presented in [5]: A cloud-based model and an optimized model for edge computing. end if 15: end while Cloud-based Model.…”

“…For the cloud-based implementation, we used a MacBook Pro with an Apple M1 Pro chip. We used two pre-trained models in [5] as tiny machine learning and cloud models. Figure 4 (b) and (c) depict two different setups for the power measurement including a drone [5], [6] used in this paper and GAPuino [24], [25], [26] which has the GAP8 processor [27].…”

“…We used two pre-trained models in [5] as tiny machine learning and cloud models. Figure 4 (b) and (c) depict two different setups for the power measurement including a drone [5], [6] used in this paper and GAPuino [24], [25], [26] which has the GAP8 processor [27]. After extracting offline power consumption and latency, we used them in the simulation while implementing the proposed algorithm in this paper.…”

“…Tiny drones have equipped with various sensors such as LIDAR and cameras and the collected data by these sensors can be fed to ML Neural Networks (NNs). Vision-based Deep Neural Networks (DNNs) [3], [5], [4] or Reinforcement Learning (RL) [6], [7], [8], [9], [10] approaches can be deployed on such tiny drones to enable them to perform complex tasks. Due to the intensive computational requirements of DNNs models, cloud-based approaches which provide un-limited computational capacity have been highly addressed in this area.…”

“…Fig. 4: (a) Crazyflie with AI-Deck, (b) drone power measurement setup[5],[6], and (c) power measurement setup includes GAPuino with camera[24].…”