Deep Multi-view Depth Estimation with Predicted Uncertainty

Abstract

In this paper, we address the problem of estimating dense depth from a sequence of images using deep neural networks. Specifically, we employ a dense-optical-flow network to compute correspondences and then triangulate the point cloud to obtain an initial depth map. Parts of the point cloud, however, may be less accurate than others due to lack of common observations or small baseline-to-depth ratio.To further increase the triangulation accuracy, we introduce a depth-refinement network (DRN) that optimizes the initial depth map based on the image's contextual cues. In particular, the DRN contains an iterative refinement module (IRM) that improves the depth accuracy over iterations by refining the deep features. Lastly, the DRN also predicts the uncertainty in the refined depths, which is desirable in applications such as measurement selection for scene reconstruction. We show experimentally that our algorithm outperforms state-of-the-art approaches in terms of depth accuracy, and verify that our predicted uncertainty is highly correlated to the actual depth error.

Learning Optical Flow, Depth, and Scene Flow without Real-World Labels

“…This predicted optical flow is used to generate a depth map via triangulation [16]. For every pixel x t i = [x t i , y t i , 1] T in frame t, its predicted optical flow corresponds to the displacement ôi between frames I t and I t+1 , such that x t+1 i = x t i + ôi .…”

“…Using geometric view synthesis as a learning objective, this approach has been successfully applied to a wide range of key robot vision tasks in the challenging monocular setting, including the estimation of ego-motion [1], [2], the 6 degree-of-freedom camera translation and rotation; depth [3], [4], [5], [6], the perpixel distance value from the image plane; optical flow [7], [8], [9], [10], [11], the 2D pixel displacement between frames; and scene flow [12], [13], [14], [15], the 3D motion of each point in the scene. Although these tasks are clearly related [12], [13], [16], they typically require stereo pairs at training time [14], [15] to resolve reprojection ambiguities in self-supervision.…”