Real-time Monocular Depth Estimation with Sparse Supervision on Mobile

Yucel, Mehmet Kerim; Dimaridou, Valia; Drosou, Anastasios; Saà-Garriga, Albert

doi:10.1109/cvprw53098.2021.00275

Cited by 15 publications

(6 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FastDepth achieves 178 fps on the 8GB NVIDIA Jetson TX2 device. Recently, Yucel et al [34] propose a small network composed by the MobileNet v2 [33] as encoder and FBNet x112 [35] as decoder, trained on an altered knowledge distillation process; the model achieves 37 fps on smartphone GPU. Papa et al [7] design SPEED, a separable pyramidal pooling architecture characterized by an improved version of the MobileNet v1 [36] as an encoder and a dedicated decoder.…”

Section: Lightweight Mde Methodsmentioning

confidence: 99%

See 1 more Smart Citation

METER: A Mobile Vision Transformer Architecture for Monocular Depth Estimation

Papa

Russo

Amerini

2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Depth estimation is a fundamental knowledge for autonomous systems that need to assess their own state and perceive the surrounding environment. Deep learning algorithms for depth estimation have gained significant interest in recent years, owing to the potential benefits of this methodology in overcoming the limitations of active depth sensing systems. Moreover, due to the low cost and size of monocular cameras, researchers have focused their attention on monocular depth estimation (MDE), which consists in estimating a dense depth map from a single RGB video frame. State of the art MDE models typically rely on vision transformers (ViT) architectures that are highly deep and complex, making them unsuitable for fast inference on devices with hardware constraints.Purposely, in this paper, we address the problem of exploiting ViT in MDE on embedded devices. Those systems are usually characterized by limited memory capabilities and low-power CPU/GPU. We propose METER, a novel lightweight vision transformer architecture capable of achieving state of the art estimations and low latency inference performances on the considered embedded hardwares: NVIDIA Jetson TX1 and NVIDIA Jetson Nano. We provide a solution consisting of three alternative configurations of METER, a novel loss function to balance pixel estimation and reconstruction of image details, and a new data augmentation strategy to improve the overall final predictions. The proposed method outperforms previous lightweight works over the two benchmark datasets: the indoor NYU Depth v2 and the outdoor KITTI.

show abstract

Section: Lightweight Mde Methodsmentioning

confidence: 99%

“…In this section, METER is compared with state of the art lightweight models as [7], [8], [31], [32], [34], which are designed to infer at high speed on embedded devices while keeping a small memory footprint (lower than 3GB). This choice is due to the limited amount of available memory in the chosen platforms.…”

Section: A Comparison With State Of the Art Methodsmentioning

confidence: 99%

METER: A Mobile Vision Transformer Architecture for Monocular Depth Estimation

Papa

Russo

Amerini

2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Real-time depth estimators are often motivated by other applications like robotics [25], automated driving [64], or low-resolution estimation for augmented reality in mobile devices [63], therefore, do not yet provide sufficient performance for immersive video. Yet, deep-learning based monocular depth estimation is also progressing quickly [62], [65], indicating the possibility of using them in near future also for immersive video applications.…”

Section: B Depth Estimation For Immersive Videomentioning

confidence: 99%

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Mieloch¹,

Garus

Milovanović

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

This paper presents the overview and rationale behind the Decoder-Side Depth Estimation (DSDE) mode of the MPEG Immersive Video (MIV) standard, using the Geometry Absent profile, for efficient compression of immersive multiview video. A MIV bitstream generated by an encoder operating in the DSDE mode does not include depth maps. It only contains the information required to reconstruct them in the client or in the cloud: decoded views and metadata. The paper explains the technical details and techniques supported by this novel MIV DSDE mode. The description additionally includes the specification on Geometry Assistance Supplemental Enhancement Information which helps to reduce the complexity of depth estimation, when performed in the cloud or at the decoder side. The depth estimation in MIV is a non-normative part of the decoding process, therefore, any method can be used to compute the depth maps. This paper lists a set of requirements for depth estimation, induced by the specific characteristics of the DSDE. The depth estimation reference software, continuously and collaboratively developed with MIV to meet these requirements, is presented in this paper. Several original experimental results are presented. The efficiency of the DSDE is compared to two MIV profiles. The combined non-transmission of depth maps and efficient coding of textures enabled by the DSDE leads to efficient compression and rendering quality improvement compared to the usual encoder-side depth estimation. Moreover, results of the first evaluation of state-of-the-art multiview depth estimators in the DSDE context, including machine learning techniques, are presented.

show abstract

“…One simple alternative is employing lightweight architectures such as MobileNet [24,25,49,55], GhostNet [21], and FBNet [54]. One popular approach is utilizing network compression techniques, including quantization [22], network pruning [58], and knowledge distillation [60]. Other methods employ well-known pyramid networks or dynamic optimization schemes [1].…”

Section: Related Workmentioning

confidence: 99%

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Huynh

Nguyen-Ha

Matas

et al. 2022

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

This paper presents a novel neural architecture search method, called LiDNAS, for generating lightweight monocular depth estimation models. Unlike previous neural architecture search (NAS) approaches, where finding optimized networks is computationally demanding, the introduced novel Assisted Tabu Search leads to efficient architecture exploration. Moreover, we construct the search space on a pre-defined backbone network to balance layer diversity and search space size. The LiDNAS method outperforms the state-of-the-art NAS approach, proposed for disparity and depth estimation, in terms of search efficiency and output model performance. The LiDNAS optimized models achieve result superior to compact depth estimation state-of-the-art on NYU-Depth-v2, KITTI, and Scan-Net, while being 7%-500% more compact in size, i.e the number of model parameters.

show abstract

Real-time Monocular Depth Estimation with Sparse Supervision on Mobile

Cited by 15 publications

References 39 publications

METER: A Mobile Vision Transformer Architecture for Monocular Depth Estimation

METER: A Mobile Vision Transformer Architecture for Monocular Depth Estimation

Overview and Efficiency of Decoder-Side Depth Estimation in MPEG Immersive Video

Lightweight Monocular Depth with a Novel Neural Architecture Search Method

Contact Info

Product

Resources

About