Moving Indoor: Unsupervised Video Depth Learning in Challenging Environments

Abstract

Recently unsupervised learning of depth from videos has made remarkable progress and the results are comparable to fully supervised methods in outdoor scenes like KITTI. However, there still exist great challenges when directly applying this technology in indoor environments, e.g., large areas of non-texture regions like white wall, more complex ego-motion of handheld camera, transparent glasses and shiny objects. To overcome these problems, we propose a new optical-flow based training paradigm which reduces the difficulty of unsupervised learning by providing a clearer training target and handles the non-texture regions. Our experimental evaluation demonstrates that the result of our method is comparable to fully supervised methods on the NYU Depth V2 benchmark. To the best of our knowledge, this is the first quantitative result of purely unsupervised learning method reported on indoor datasets.

Video Autoencoder: self-supervised disentanglement of static 3D structure and motion

“…(i) SFMLearner (and similarly, GeoNet, etc.) results were shown on the relatively simple KITTI dataset, but work poorly on more complex data [78] because its spatial representation is a 2.5D depth map. We use the predicted depth and camera transformation to warp the first frame into the target frame.…”

PLNet: Plane and Line Priors for Unsupervised Indoor Depth Estimation

“…the indoor scenario, and only a few attempts have been made. As pointed out by the pioneer work [59], indoor videos, such as the NYU Depth V2 dataset [43], have complicated ego-motion, as they are usually recorded by handheld cameras. The problem can be alleviated by sampling the more distant (±10) frames as the source frames [59] or weakly rectifying the training sequences [3].…”

“…As pointed out by the pioneer work [59], indoor videos, such as the NYU Depth V2 dataset [43], have complicated ego-motion, as they are usually recorded by handheld cameras. The problem can be alleviated by sampling the more distant (±10) frames as the source frames [59] or weakly rectifying the training sequences [3]. Alternatively, we could construct a dataset by moving the camera steadily and sufficiently to solve the problem.…”

“…Adequate research has been done for this difficulty [53,2,4,38,18,31,27,28,33]. However, only a few works began to apply the unsupervised framework to the more challenging indoor scene [59,57,3,54]. To learn the correspondence well for the textureless regions, Zhou et al [59] proposed to predict optical flow in a sparseto-dense propagation manner and use the optical flow to guide the rigid flow generated by the unsupervised depth learning framework.…”

“…However, only a few works began to apply the unsupervised framework to the more challenging indoor scene [59,57,3,54]. To learn the correspondence well for the textureless regions, Zhou et al [59] proposed to predict optical flow in a sparseto-dense propagation manner and use the optical flow to guide the rigid flow generated by the unsupervised depth learning framework. To learn the ego-motion better, Train-Flow [57] estimates the pose by reliable correspondences from a FlowNet with a differentiable two-view triangulation module.…”