P3Depth: Monocular Depth Estimation with a Piecewise Planarity Prior

Abstract

Monocular depth estimation is vital for scene understanding and downstream tasks. We focus on the supervised setup, in which ground-truth depth is available only at training time. Based on knowledge about the high regularity of real 3D scenes, we propose a method that learns to selectively leverage information from coplanar pixels to improve the predicted depth. In particular, we introduce a piecewise planarity prior which states that for each pixel, there is a seed pixel which shares the same planar 3D surface with the former. Motivated by this prior, we design a network with two heads. The first head outputs pixel-level plane coefficients, while the second one outputs a dense offset vector field that identifies the positions of seed pixels. The plane coefficients of seed pixels are then used to predict depth at each position. The resulting prediction is adaptively fused with the initial prediction from the first head via a learned confidence to account for potential deviations from precise local planarity. The entire architecture is trained end-to-end thanks to the differentiability of the proposed modules and it learns to predict regular depth maps, with sharp edges at occlusion boundaries. An extensive evaluation of our method shows that we set the new state of the art in supervised monocular depth estimation, surpassing prior methods on NYU Depth-v2 and on the Garg split of KITTI. Our method delivers depth maps that yield plausible 3D reconstructions of the input scenes. Code is available at: https://github.com/SysCV/P3Depth

Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention

“…The performance gain is significant considering the saturated performance of the dataset in recent years. We see an improvement of 9.6% and 3.5% over the recently proposed methods [23] and NeWCRFs, respectively, in terms of the RMSE error. We see in Fig.…”

“…Lee et al [17] enforce a model to learn structural information about the scene by learning the relationship between image patches close to each other. Whereas, Patil et al [23] exploit coplanar pixels to improve the predicted depth.…”

DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimation

“…Recently, transformer structures have been introduced in monocular depth estimation, where DPT [43], and PixelFormer [1] have shown improved performances. To increase the robustness of monocular depth estimation, some methods introduce additional constraints such as uncertainty (UCRDepth [47]) or piecewise planarity prior (P3Depth [41]). NewCRFs [64] proposes windowseparated Conditional Random Fields (CRF) to enhance the local space relation with neighboring pixels.…”

“…Estimating depth from a single image is challenging due to the inherent ambiguity in the mapping between the 2D image and the 3D scene. To increase the robustness, the following methods utilizing constructed additional constraints such as uncertainty (UCRDepth [47]), and piecewise planarity prior (P3Depth [41]). The NewCRFs [64] introduces window-separated Conditional Random Fields (CRF) to enhance local space relation with neighbor pixels.…”

SD-Depth: Light-Weight Monocular Depth Estimation Using Space Depth CNN for Real-Time Applications

“…An encoder-decoder transformer-based architectural block that separates the depth range into bins with estimated centers depending on the image was proposed by Bhat et al [11]. Patil et al [12] recently proposed a two head CNN network for depth estimation, the first head produces pixel-wise plane values while the second head produces a dense vector field for each pixel position. The output from the second head is adaptively fused by the output of the first head.…”