OANet: Learning Two-View Correspondences and Geometry Using Order-Aware Network

Zhang, Jiahui; Sun, Dawei; Luo, Zixin; Yao, Anbang; Chen, Hongkai; Zhou, Lei; Shen, Tianwei; Chen, Yurong; Quan, Long; Liao, Hongen

doi:10.1109/tpami.2020.3048013

Cited by 63 publications

(151 citation statements)

References 73 publications

Supporting

Mentioning

151

Contrasting

Order By: Relevance

“…Alternatively to making RANSAC differentiable, some authors propose to replace RANSAC by a neural network [61], [62], [63], [64], [65]. In these works, the neural network acts as a classifier for model inliers, effectively acting as a robust estimator for model parameters.…”

Section: Differentiable Robust Estimatorsmentioning

confidence: 99%

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

Brachmann¹,

Rother

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

We describe a learning-based system that estimates the camera position and orientation from a single input image relative to a known environment. The system is flexible w.r.t. the amount of information available at test and at training time, catering to different applications. Input images can be RGB-D or RGB, and a 3D model of the environment can be utilized for training but is not necessary. In the minimal case, our system requires only RGB images and ground truth poses at training time, and it requires only a single RGB image at test time. The framework consists of a deep neural network and fully differentiable pose optimization. The neural network predicts so called scene coordinates, i.e. dense correspondences between the input image and 3D scene space of the environment. The pose optimization implements robust fitting of pose parameters using differentiable RANSAC (DSAC) to facilitate end-to-end training. The system, an extension of DSAC++ and referred to as DSAC*, achieves state-of-the-art accuracy on various public datasets for RGB-based re-localization, and competitive accuracy for RGB-D based re-localization.

show abstract

Section: Differentiable Robust Estimatorsmentioning

confidence: 99%

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

Brachmann¹,

Rother

2021

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…For the handcrafted methods, two variations of nearest neighbor (NN) matching, the mutual NN with PyTorch implementation, and the FLANN with OpenCV implementation when the test ratio is 0.7, were evaluated. For the learned methods, the PointCN [16], OANet [18], ACNe [19], and SuperGlue [28] were assessed. During testing, the official released code and model weights were used.…”

Section: Comparison To Related Work 1) Setupsmentioning

confidence: 99%

“…Given the predicted keypoint matches of an image pair, the essential matrix is obtained with OpenCV function findEssentialMat, and then the relative pose is recovered with recoverPose. As in previous works [16], [18], [28], we calculate the pose angular differences between ground truth and estimated pose, and report the area under curve (AUC) with a maximum error of threshold 5°, 10°, and 20°.…”

Section: ) Pose Estimation Accuracymentioning

confidence: 99%

See 1 more Smart Citation

Probabilistic Spatial Distribution Prior Based Attentional Keypoints Matching Network

Zhao

Liu

et al. 2022

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Keypoints matching is a pivotal component for many image-relevant applications such as image stitching, visual simultaneous localization and mapping (SLAM), and so on. Both handcrafted-based and recently emerged deep learning-based keypoints matching methods merely rely on keypoints and local features, while losing sight of other available sensors such as inertial measurement unit (IMU) in the above applications. In this paper, we demonstrate that the motion estimation from IMU integration can be used to exploit the spatial distribution prior of keypoints between images. To this end, a probabilistic perspective of attention formulation is proposed to integrate the spatial distribution prior into the attentional graph neural network naturally. With the assistance of spatial distribution prior, the effort of the network for modeling the hidden features can be reduced. Furthermore, we present a projection loss for the proposed keypoints matching network, which gives a smooth edge between matching and un-matching keypoints. Image matching experiments on visual SLAM datasets indicate the effectiveness and efficiency of the presented method.

show abstract

“…However, this study could not take advantage of the relative motion pose shared by neighboring pixels [ 53 ]. Zhang et al [ 54 ] used neural networks to infer the probability of each corresponding point as an interior point and then restored the camera pose. Tabb et al [ 55 ] used rigid constraints to represent the camera network and multi-camera calibration problem and expressed it as a system of equations to obtain approximate solutions.…”

Section: Related Workmentioning

confidence: 99%

Parallel Structure from Motion for Sparse Point Cloud Generation in Large-Scale Scenes

Bao

Lin

et al. 2021

Sensors

View full text Add to dashboard Cite

Scene reconstruction uses images or videos as input to reconstruct a 3D model of a real scene and has important applications in smart cities, surveying and mapping, military, and other fields. Structure from motion (SFM) is a key step in scene reconstruction, which recovers sparse point clouds from image sequences. However, large-scale scenes cannot be reconstructed using a single compute node. Image matching and geometric filtering take up a lot of time in the traditional SFM problem. In this paper, we propose a novel divide-and-conquer framework to solve the distributed SFM problem. First, we use the global navigation satellite system (GNSS) information from images to calculate the GNSS neighborhood. The number of images matched is greatly reduced by matching each image to only valid GNSS neighbors. This way, a robust matching relationship can be obtained. Second, the calculated matching relationship is used as the initial camera graph, which is divided into multiple subgraphs by the clustering algorithm. The local SFM is executed on several computing nodes to register the local cameras. Finally, all of the local camera poses are integrated and optimized to complete the global camera registration. Experiments show that our system can accurately and efficiently solve the structure from motion problem in large-scale scenes.

show abstract

OANet: Learning Two-View Correspondences and Geometry Using Order-Aware Network

Cited by 63 publications

References 73 publications

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC

Probabilistic Spatial Distribution Prior Based Attentional Keypoints Matching Network

Parallel Structure from Motion for Sparse Point Cloud Generation in Large-Scale Scenes

Contact Info

Product

Resources

About