Learning to Find Good Correspondences

Abstract

We develop a deep architecture to learn to find good correspondences for wide-baseline stereo. Given a set of putative sparse matches and the camera intrinsics, we train our network in an end-to-end fashion to label the correspondences as inliers or outliers, while simultaneously using them to recover the relative pose, as encoded by the essential matrix. Our architecture is based on a multi-layer perceptron operating on pixel coordinates rather than directly on the image, and is thus simple and small. We introduce a novel normalization technique, called Context Normalization, which allows us to process each data point separately while embedding global information in it, and also makes the network invariant to the order of the correspondences. Our experiments on multiple challenging datasets demonstrate that our method is able to drastically improve the state of the art with little training data.

Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses

“…Besides DSAC, a differentiable robust estimator, there has recently been some work on learning robust estimators. We discussed the work of Yi et al [56] in the introduction. Ranftl and Koltun [36] take a similar but iterative approach reminiscent of Iteratively Reweighted Least Squares (IRLS) for fundamental matrix estimation.…”

“…The residuals of the last iteration are an additional input to the network in the next iteration. The network architecture is similar to the one used in [56]. Correspondences are represented as 4D vectors, and they use the descriptor matching ratio as an additional input.…”

“…• brown cogsci 2 -brown cogsci 2 • brown cogsci 6 -brown cogsci 6 • brown cogsci 8 -brown cogsci 8 • brown cs 3 -brown cs3 • brown cs 7 -brown cs7 • harvard c4 -hv c4 1 • harvard c10 -hv c10 2 • harvard corridor lounge -hv lounge1 2 • harvard robotics lab -hv s1 2 • hotel florence jx -florence hotel stair room all • mit 32 g725 -g725 1 • mit 46 6conf -bcs floor6 conf 1 • mit 46 6lounge -bcs floor6 long • mit w85g -g 0 • mit w85h -h2 1 Network Architecture. As mentioned in the main paper, we replicated the architecture of Yi et al [56] for our experiments on epipolar geometry (estimating essential and fun-damental matrices). For a schematic overview see Fig.…”

“…The arc with a plus marks a skip connection for each of the twelve blocks [18]. This architecture was proposed by Yi et al [56].…”

“…a validation set would not be strictly required. When calculating the AUC for evaluation, we adhere to the protocol of Yi et al [56] to ensure comparability. Yi et al approximate the AUC via the area under the cumulative histogram with a bin width of 5 • .…”