A Two-Stage Deep Learning Registration Method for Remote Sensing Images Based on Sub-Image Matching

Abstract

The registration of multi-temporal remote sensing images with abundant information and complex changes is an important preprocessing step for subsequent applications. This paper presents a novel two-stage deep learning registration method based on sub-image matching. Unlike the conventional registration framework, the proposed network learns the mapping between matched sub-images and the geometric transformation parameters directly. In the first stage, the matching of sub-images (MSI), sub-images cropped from the images are matched through the corresponding heatmaps, which are made of the predicted similarity of each sub-image pairs. The second stage, the estimation of transformation parameters (ETP), a network with weight structure and position embedding estimates the global transformation parameters from the matched pairs. The network can deal with an uncertain number of matched sub-image inputs and reduce the impact of outliers. Furthermore, the sample sharing training strategy and the augmentation based on the bounding rectangle are introduced. We evaluated our method by comparing the conventional and deep learning methods qualitatively and quantitatively on Google Earth, ISPRS, and WHU Building Datasets. The experiments showed that our method obtained the probability of correct keypoints (PCK) of over 99% at α = 0.05 (α: the normalized distance threshold) and achieved a maximum increase of 16.8% at α = 0.01, compared with the latest method. The results demonstrated that our method has good robustness and improved the precision in the registration of optical remote sensing images with great variation.

Evaluation of Simulated CO2 Point Source Plumes from High-Resolution Atmospheric Transport Model

“…When extracting positive samples from target images, these windows can either fit within the bounding rectangles or entirely contain them, as depicted in Figure 9c,d. As a result, in order to address this, we introduce a method to enhance training by augmenting target sub-images [51]. Instead of relying on exact centers transformed from the source sub-images, random cropping within this predefined scope is employed.…”