Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection

Ye, Nianjin; Wang, Chuan; Fan, Haoqiang; Liu, Shuaicheng

doi:10.1109/iccv48922.2021.01287

Cited by 39 publications

(33 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dataset Following [35] and [39], we evaluate our method on a natural image dataset [39] with 75.8k training pairs and 4.2k testing pairs of image size 320 × 640. In both subsets, the image pairs are roughly evenly categorized into five types of scenes, respectively are regular (RE), low texture (LT), low light (LL), small foreground (SF), and large foreground (LF), where the last four are challenging scenes for homography estimation.…”

Section: Methodsmentioning

confidence: 99%

“…If learning from synthetic images, the lack of realistic transformation will degrade their generalization ability. Unsupervised methods [25,35,39] typically optimize their model by minimizing a distance from the source image warped by the predicted homography to the target image. [39] and [18] introduced mask prediction into homography estimation, but their goal is to remove large foregrounds or moving objects, while our goal is to preserve a single dominant plane with explicit constraint.…”

Section: Related Workmentioning

confidence: 99%

“…Given a pair of gray-scale image patches I a and I b of size H × W , we predict the homography transformation from I a to I b , denoted as H ab . Following [35], we decompose a homography matrix into 8 orthogonal flow bases, and predict the weights of 8 bases instead of regressing the homography matrix or the corner offsets [8,25,39]. The pipeline of our method is illustrated in Fig.…”

Section: Overviewmentioning

confidence: 99%

“…The second term is a feature identity loss L F IL that lets the feature projector F be warp-equivalent [35], which is written as:…”

Section: Network Trainingmentioning

confidence: 99%

“…Recently, unsupervised learning methods have gained popularity in homography estimation [8,25,28,35,39]. These methods directly predict the homography from a pair of source and target images using a neural network, of which an important optimization objective is to minimize the distance from the warped source image to the target image.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Unsupervised Homography Estimation with Coplanarity-Aware GAN

Hong¹,

Lu²,

Ye³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Estimating homography from an image pair is a fundamental problem in image alignment. Unsupervised learning methods have received increasing attention in this field due to their promising performance and label-free training. However, existing methods do not explicitly consider the problem of plane-induced parallax, which will make the predicted homography compromised on multiple planes. In this work, we propose a novel method HomoGAN to guide unsupervised homography estimation to focus on the dominant plane. First, a multi-scale transformer network is designed to predict homography from the feature pyramids of input images in a coarse-to-fine fashion. Moreover, we propose an unsupervised GAN to impose coplanarity constraint on the predicted homography, which is realized by using a generator to predict a mask of aligned regions, and then a discriminator to check if two masked feature maps are induced by a single homography. To validate the effectiveness of HomoGAN and its components, we conduct extensive experiments on a large-scale dataset, and results show that our matching error is 22% lower than the previous SOTA method. Code is available at https: //github.com/megvii-research/HomoGAN .

show abstract

Section: Methodsmentioning

confidence: 99%