Multilevel Pyramid Network for Monocular Depth Estimation Based on Feature Refinement and Adaptive Fusion

Abstract

As a traditional computer vision task, monocular depth estimation plays an essential role in novel view 3D reconstruction and augmented reality. Convolutional neural network (CNN)-based models have achieved good performance for this task. However, in the depth map recovered by some existing deep learning-based methods, local details are still lost. To generate convincing depth maps with rich local details, this study proposes an efficient multilevel pyramid network for monocular depth estimation based on feature refinement and adaptive fusion. Specifically, a multilevel spatial feature generation scheme is developed to extract rich features from the spatial branch. Then, a feature refinement module that combines and enhances these multilevel contextual and spatial information is designed to derive detailed information. In addition, we design an adaptive fusion block for improving the capability of fully connected features. The performance evaluation results on public RGBD datasets indicate that the proposed approach can recover reasonable depth outputs with better details and outperform several depth recovery algorithms from a qualitative and quantitative perspective.

URNet: An UNet-Based Model with Residual Mechanism for Monocular Depth Estimation

“…Alternatively, numerous researchers employed labeled images as the input data in the training procedure [30,[36][37][38][39][40], where the models can achieve superior performance, depending on the quality of input data. More specifically, several researchers proposed their new model for monocular depth estimation using multiple-scale features [30,41,42], and the performance of these models is exceptional.…”

“…More specifically, several researchers proposed their new model for monocular depth estimation using multiple-scale features [30,41,42], and the performance of these models is exceptional. Next, Xu et al in [40] proposed a pyramid network with the concept of an adaptive fusion block to improve the exploiting capability of the input depth map and the depth estimation performance. DPNet [37] was a comprehensive solution to the problems of inaccurate depth inference and the loss of spatial information, where both the contextual branch and spatial branch were described in detail.…”

“…On the other hand, Table 2 displays the comprehensive results and comparisons among various depth recovery algorithms and ours, where parts of the materials are from [40]. In this table, the type "Stereo" indicates self-supervised learning by utilizing stereo supervision, and the type "Depth" indicates the supervised learning methods by utilizing depth supervision.…”

“…In this table, the type "Stereo" indicates self-supervised learning by utilizing stereo supervision, and the type "Depth" indicates the supervised learning methods by utilizing depth supervision. Our method is compared to these existing works [14][15][16][30][31][32][34][35][36][37][38]40,41]. More specifically, consider the first three comparisons, RMSE, MRE, and RMLSE, where the lower the better.…”