Monocular Depth Perception on Microcontrollers for Edge Applications

Abstract

Depth estimation is crucial in several computer vision applications, and a recent trend in this field aims at inferring such a cue from a single camera. Unfortunately, despite the compelling results achieved, state-of-the-art monocular depth estimation methods are computationally demanding, thus precluding their practical deployment in several application contexts characterized by low-power constraints. Therefore, in this paper, we propose a lightweight Convolutional Neural Network based on a shallow pyramidal architecture, referred to as µPyD-Net, enabling monocular depth estimation on microcontrollers. The network is trained in a peculiar self-supervised manner leveraging proxy labels obtained through a traditional stereo algorithm. Moreover, we propose optimization strategies aimed at performing computations with quantized 8-bit data and map the high-level description of the network to low-level layers optimized for the target microcontroller architecture. Exhaustive experimental results on standard datasets and an in-depth evaluation with a device belonging to the popular Arm Cortex-M family confirm that obtaining sufficiently accurate monocular depth estimation on microcontrollers is feasible. To the best of our knowledge, our proposal is the first one enabling such remarkable achievement, paving the way for the deployment of monocular depth cues onto the tiny end-nodes of distributed sensor networks.

Investigating data representation for efficient and reliable Convolutional Neural Networks

“…Nowadays, to address data confidentiality issues and bandwidth limitations, the trend is to push deep learning based systems from the cloud to edge devices [15,16,17], such as Internet-of-Things (IoTs) devices, given the ever-increasing internet-connected IoTs. One of the principal advantage is that it alleviates the communication latency which is unacceptable for real-time safety-critical decisions, e.g., in autonomous driving.…”

Application of Machine Vision Techniques in Low-Cost Devices to Improve Efficiency in Precision Farming

“…Further, Peluso et al [47] (2022) propose an efficient monocular depth estimation method for microcontrollers based on a lightweight CNN with a shallow pyramidal architecture. By using optimization strategies to perform calculations on 8-bit data and mapping the high-level description of the network to low-level layers optimized for the target microcontroller architecture, experimental results show that it is possible to obtain depth estimates sufficiently accurate for objects with large overlap areas.…”

Lightweight and Energy-Aware Monocular Depth Estimation Models for IoT Embedded Devices: Challenges and Performances in Terrestrial and Underwater Scenarios

“…The model is aimed to overcome cutting-edge design difficulties, which are often deep and complicated, requiring dedicated hardware for their execution such as high-end and power-hungry GPUs. Peluso et al., on their side, propose [ 8 , 10 ]. The first work presents a framework for optimizing inference performance in order to produce a low-latency/high-throughput code.…”

“…Regarding monocular depth estimation, only a few works propose a solution for porting such complex tasks on low-resource platforms. There are two main approaches: [ 8 , 9 , 10 ] that focus on MDE on microcontroller and ARM-powered devices without taking into account the inference frequency and [ 11 , 12 , 13 ], which analyze the inference performances of MDE on low-power embedded GPUs. Furthermore, MDE methods are usually trained in supervised learning strategies on indoor and outdoor terrestrial datasets such as [ 14 , 15 ].…”