Content-Aware Unsupervised Deep Homography Estimation

Abstract

Robust homography estimation between two images is a fundamental task which has been widely applied to various vision applications. Traditional feature based methods often detect image features and fit a homography according to matched features with RANSAC outlier removal. However, the quality of homography heavily relies on the quality of image features, which are prone to errors with respect to low light and low texture images. On the other hand, previous deep homography approaches either synthesize images for supervised learning or adopt aerial images for unsupervised learning, both ignoring the importance of depth disparities in homography estimation. Moreover, they treat the image content equally, including regions of dynamic objects and near-range foregrounds, which further decreases the quality of estimation. In this work, to overcome such problems, we propose an unsupervised deep homography method with a new architecture design. We learn a mask during the estimation to reject outlier regions. In addition, we calculate loss with respect to our learned deep features instead of directly comparing the image contents as did previously. Moreover, a comprehensive dataset is presented, covering both regular and challenging cases, such as poor textures and non-planar interferences. The effectiveness of our method is validated through comparisons with both feature-based and previous deep-based methods. Code will be soon available at Github 1 .

Road-aware Monocular Structure from Motion and Homography Estimation

“…Finally, image pairs with inlier correspondences greater than 50 and keypoints evenly distributed will be selected to constitute the validation dataset. Similar to existing methods [46,25] the reprojection errors of the correspondences are utilized 2. Depth evaluation results on the KITTI Eigen split [9].…”

“…Nonetheless, the performance may degrade dramatically when the detected keypoints are insufficient or distribute unevenly, which is common phenomenon in driving scenes. Recently, inspired by the success of deep convolution neural network (CNN) in computer vision, deep homography estimation methods become prevalent [7,33,46,25]. These methods directly regress the coordinates offset of specified 4 points [7] according to the 4-points parameter estimation.…”

“…CNN-based methods treat homography estimation as a regression problem via the 4-point parameterization [7,33,46]. By leveraging high level context features, these methods often outperform the traditional methods on challenging cases mentioned above.…”

TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations

“…Sparse local features like SIFT [33] can be matched either using nearest neighbour, or deep models like OANet [72] and SuperGlue [47], and the resulting correspondences can be used to estimate warping models. Recently, deep models have been explored to directly learn homography parameters [8,73,38], demonstrating their advantages on low-light and low-texture images.…”

DeepMeshFlow: Content Adaptive Mesh Deformation for Robust Image Registration

“…Recently, deep homography has been proposed which takes two images as input to the network and output the homography [12,8]. Compared with feature-based methods, deep homography is more robust against various challenging cases, such as low-light, low-texture, highnoise, etc [13]. The other problem of homography is its limited degree of freedom.…”