Toward Domain Independence for Learning-Based Monocular Depth Estimation

Abstract

Modern autonomous mobile robots require a strong understanding of their surroundings in order to safely operate in cluttered and dynamic environments. Monocular depth estimation offers a geometry-independent paradigm to detect free, navigable space with minimum space, and power consumption. These represent highly desirable features, especially for microaerial vehicles. In order to guarantee robust operation in real-world scenarios, the estimator is required to generalize well in diverse environments. Most of the existent depth estimators do not consider generalization, and only benchmark their performance on publicly available datasets after specific fine tuning. Generalization can be achieved by training on several heterogeneous datasets, but their collection and labeling is costly. In this letter, we propose a deep neural network for scene depth estimation that is trained on synthetic datasets, which allow inexpensive generation of ground truth data. We show how this approach is able to generalize well across different scenarios. In addition, we show how the addition of long short-term memory layers in the network helps to alleviate, in sequential image streams, some of the intrinsic limitations of monocular vision, such as global scale estimation, with low computational overhead. We demonstrate that the network is able to generalize well with respect to different real-world environments without any fine tuning, achieving comparable performance to state-of-the-art methods on the KITTI dataset.

DroNet: Learning to Fly by Driving

“…Another promising approach has been use simulations to get training data for reinforcement or imitation learning tasks, while testing the learned policy in the real world [14], [15], [11]. Clearly, this approach suffers from the domain shift between simulation and reality and might require some realworld data to be able to generalize [11].…”

“…Indeed, we conjecture the network to make decision on the base of the distance to observed objects in the field of view. Convolutional networks are in fact well known to be successful on the task of monocular depth estimation [15]. An interesting question that we would like to answer in future work is how this approach compares to an LSTM [22] based solution, making decisions over a temporal horizon.…”

Fusion of Stereo and Still Monocular Depth Estimates in a Self-Supervised Learning Context

“…Out of the 16 layers of the truncated VGG network, the first 8 were kept fixed, while the others were finetuned for the task of depth estimation. In order to accomodate for this task, in [20] two deconvolutional layers were added to the network that bring the neural representation back to the desired depth map resolution.…”

“…In [20], the FCN was trained on depth maps obtained from various visually highly realistic simulated environments. In the current study, we will train and fine-tune the same layers, but then using sparse stereo-based disparity measurements as supervised targets.…”

An educational simulation platform for GPS-denied unmanned Aerial Vehicles aimed to the detection and tracking of moving objects

“…Different solutions, typically based on external robotic simulators such as Gazebo [11], V-REP [12], AirSim [13], MORSE [14], are available to this aim. They employ recent advances in computation and computer graphics (e.g., AirSim is a photorealistic environment [7]) in order to simulate physical phenomena (gravity, magnetism, atmospheric conditions) and perception (e.g., providing sensor models) in such a way that the environment realistically reflects the actual world. In some cases those solutions do not have enough features that could allow to create large scale complex environments close to reality.…”