Fast Uncertainty Estimation for Deep Learning Based Optical Flow

Abstract

We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based on Bayesian neural networks have been proposed to handle this issue, they are computationally expensive. Thus, to speed up the processing time, our approach applies a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach. By using synthetically generated images of spacecraft, we demonstrate that the trained generative model can produce the uncertainty map 100∼700 times faster than the conventional uncertainty estimation method used for training the generative model itself. We also show that the quality of uncertainty map derived by the generative model is close to that of the original uncertainty map. By applying the proposed approach, the deep learning model operated in real-time can avoid disastrous failures by considering the uncertainty as well as achieving better performance removing uncertain portions of the prediction result.

Review of advanced guidance and control algorithms for space/aerospace vehicles

“…Besides, aiming at reducing the processing time required 1050 to obtain the estimation uncertainty map, a novel estimation approach based on a modified FlowNet2 network was proposed in [124]. A number of simulation studies were carried out on a spacecraft pose estimation problem and the obtained results not only confirmed the effectiveness of the proposed approach but also demonstrated good potentials for practical applications.…”

“…Specifically, four potential connections can be identified by reviewing the literature. of literature [79,[121][122][123][124][125]. Specifically, in [121], the problem of spacecraft proximity operations including formation flying and on-orbit servicing was considered.…”

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

“…The authors apply their framework to large CNNs to find the optical flow in given video sequences. Dropout-based uncertainty estimation in optical flow tasks is also considered in [416] or [777]. Building on the ADFbased framework, the authors in [455] combine assumed density filtering with dropout to further integrate epistemic uncertainty into the framework.…”

Collaborative Pose Estimation of an Unknown Target Using Multiple Spacecraft

“…When the initial frame and the inertia matrix of the target are unknown, direct observation of angular measurement has advantages such as simpler propagation of attitude quaternion and avoiding estimation of the inertia matrix. The optical flow may be obtained by classical methods such as Lucas-Kanade [28] or more recent methods such as using a convolutional neural network trained with sequences of images [29].…”

“…Chaser initial states in target-LVLH frame reader to[19],[29] for further implementation details. Future work includes validation of the algorithm with the computer vision algorithm in the loop using the realistic synthetic images.Extracted keypoints in this simulation are obtained by projecting a set of pre-defined 3D landmarks attached to the exterior of the spacecraft model to each chaser's image plane.…”