Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network

Zhang, Han; Ni, Weijian; Yan, Weidong; Xiang, Deliang; Jian, Wu; Yang, Xiu; Bu, Hui

doi:10.1109/jstars.2019.2916560

Cited by 102 publications

(50 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Table 2. The matching results (without searching) of the Siamese network [44], the pseudo-Siamese network, the 2-ch network [31], our network, and our network with the augmented loss function on all pairs without distortions. All the models were trained only on pair 1.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks

Zhu

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

We develop a deep learning-based matching method between an RGB (red, green and blue) and an infrared image that were captured from satellite sensors. The method includes a convolutional neural network (CNN) that compares the RGB and infrared image pair and a template searching strategy that searches the correspondent point within a search window in the target image to a given point in the reference image. A densely-connected CNN is developed to extract common features from different spectral bands. The network consists of a series of densely-connected convolutions to make full use of low-level features and an augmented cross entropy loss to avoid model overfitting. The network takes band-wise concatenated RGB and infrared images as the input and outputs a similarity score of the RGB and infrared image pair. For a given reference point, the similarity scores within the search window are calculated pixel-by-pixel, and the pixel with the highest score becomes the matching candidate. Experiments on a satellite RGB and infrared image dataset demonstrated that our method obtained more than 75% improvement on matching rate (the ratio of the successfully matched points to all the reference points) over conventional methods such as SURF, RIFT, and PSO-SIFT, and more than 10% improvement compared to other most recent CNN-based structures. Our experiments also demonstrated high performance and generalization ability of our method applying to multitemporal remote sensing images and close-range images.

show abstract

Section: Resultsmentioning

confidence: 99%

“…A very recent work named SFcNet [44] used the Siamese structure with shared weights for multimodel image matching. Due to the improper structure, it performed the worst and was 22.6% lower than ours on matching rate.…”

Section: Image Pairmentioning

confidence: 99%

Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks

Zhu

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Since deep learning can automatically learn high-level features in images, it has been applied to remote sensing image matching recently. Most image matching methods using deep learning are based on a Siamese network [54], [55].In addition, the GANs are applied to image matching and registration [56], [57]. One key point of these methods is to transform one image into another image by the trained GANs to eliminate the significant differences between multispectral images.…”

Section: Related Workmentioning

confidence: 99%

Automatic Matching of Multispectral Images Based on Nonlinear Diffusion of Image Structures

Ruixiang

Zhao

Zhang

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

“…The existed hand-crafted features are hard to cover all the different cases, especially for optical and SAR images with nonlinear intensity deformation [8]. While deep learning features are capable of considering the majority of changes between the reference and sensed images, including viewpoints, resolution, radiometric differences, geometric deformations and so on [9], [10], [11]. Combinations of hand-crafted features and deep learning features are proposed to absorb their respective advantages.…”

Section: Introductionmentioning

confidence: 99%

Automatic Registration of Optical and SAR Images Via Improved Phase Congruency Model

Xiang

Tao

Wang

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In this paper, we propose an automatic and efficient method to solve optical and SAR image registration using the improved phase congruency (PC) model. First, evenly-distributed keypoints are extracted from the optical images via the block harris method. Complementary grid points are then selected in image regions with poor structural information and supplemented to the keypoint set. For each keypoint, a robust feature representation that captures the local spatial relationship is proposed based on the improved PC model. Specifically, we propose to use two different PC models, the classic PC and the SAR-PC, to construct features for optical and SAR images respectively. The PC features of several directions are aggregated to construct the feature descriptors, and a similarity metric via the phase correlation of feature descriptors is obtained. The proposed similarity metric can not only find accurate correspondence but also present efficient results without presetting the size of the search region. We compare the proposed method with two baselines and stateof-the-art (SOTA) methods, i.e. OS-SIFT, HOPC, CFOG, in various scenarios. The results show that the proposed method outperforms the baselines and shows comparable performance with SOTA methods in regions with abundant structural information and better performance in regions with less structural information. Moreover, we build a high-resolution optical and SAR image matching dataset, which consists of 10,692 nonoverlapping patch-pairs of 256 × 256 pixels and 1-m resolution. Results of two benchmarks, Siamese deep matching network and conditional Generative Adversarial Networks, show that this dataset is practical and challenging.

show abstract

Registration of Multimodal Remote Sensing Image Based on Deep Fully Convolutional Neural Network

Cited by 102 publications

References 43 publications

Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks

Matching RGB and Infrared Remote Sensing Images with Densely-Connected Convolutional Neural Networks

Automatic Matching of Multispectral Images Based on Nonlinear Diffusion of Image Structures

Automatic Registration of Optical and SAR Images Via Improved Phase Congruency Model

Contact Info

Product

Resources

About