A Point Pattern Chamfer Registration of Optical and SAR Images Based on Mesh Grids

Abstract

Automatic image registration of optical-to-Synthetic aperture radar (SAR) images is difficult because of the inconsistency of radiometric and geometric properties between the optical image and the SAR image. The intensity-based methods may require many calculations and be ineffective when there are geometric distortions between these two images. The feature-based methods have high requirements on features, and there are certain challenges in feature extraction and matching. A new automatic optical-to-SAR image registration framework is proposed in this paper. First, modified holistically nested edge detection is employed to detect the main contours in both the optical and SAR images. Second, a mesh grid strategy is presented to perform a coarse-to-fine registration. The coarse registration calculates the feature matching and summarizes the preliminary results for the fine registration process. Finally, moving direct linear transformation is introduced to perform a homography warp to alleviate parallax. The experimental results show the effectiveness and accuracy of our proposed method. However, due to the different geometric and radiometric properties of SAR and optical images, to automatically register these two types of images, one must overcome many difficulties. In particular, optical images and SAR images have different geometrical characteristics. Whereas geometric distortions such as foreshortening and layover exist in SAR images, perspective and shadow exist in optical images, which cause the differences between the two types of images. In addition, optical images and SAR images have different radiometric distortion, the SAR sensor is an active remote sensing system, but the optical sensor is a passive system [4]. A large quantity of speckle noise in SAR images renders it difficult to obtain common features from a SAR image and an optical image [5]. For these reasons, the registration of optical images and SAR images has more challenges than mono-sensor image registration.The existing optical-to-SAR registration methods are mainly divided into two types: intensity-based registration methods and feature-based registration methods. Intensity-based registration methods include mutual information (MI) [6], cross-cumulative residual entropy [7] and normalized cross-correlation (NCC) [8]. Although this kind of registration method can register multi-sensor images with intensity differences, it is insensitive to the local differences between the two images and it requires many calculations [9]. Therefore, some improved intensity-based registration methods combined edges and gradient have been proposed [10][11][12]. For example, Cheah et al. [10] proposed the adaptation of MI measure which incorporates the spatial information by combining intensity and gradient information. Chen et al. [13] implemented MI through joint histogram estimation using various interpolation algorithms to complete multi-sensor and multiresolution image registration. Saidi et al. [14] proposed a refined automatic co-registration method (RA...

Edge Prior Multilayer Segmentation Network Based on Bayesian Framework

“…HED. HED is an end-to-end edge detection algorithm based on the fully convolutional neural network and deeply supervised nets, referring to article [29,30]. By relying on the weight update of convolution kernel, HED automatically learns the rich hierarchical features and determines whether the feature is important for solving the abstraction of object edges.…”

Geometrical Matching of SAR and Optical Images Utilizing ASIFT Features for SAR-based Navigation Aided Systems

“…The process of image registration refers to the alignment of two or more images of the same scene which might be obtained with the same sensor, time, and imaging conditions, as well as by different sensors and viewpoints. The process of orientating optical and SAR images is still an open issue, which creates many challenges [9][10][11][12][13][14][15][16][17][18][19]. Many issues can be dealt with in the process of the synergy of optical and SAR data, due to the great differences between passive and active remote sensing techniques.…”

“…Many issues can be dealt with in the process of the synergy of optical and SAR data, due to the great differences between passive and active remote sensing techniques. One of the issues is the problem with the speckle noise that influences the detection of robust corresponding/tie points [9,[11][12][13]15,16,18,19]. Another issue is related to the differences in the image geometry acquired from these two devices [9,[11][12][13]15,16,18,19].…”

“…One of the issues is the problem with the speckle noise that influences the detection of robust corresponding/tie points [9,[11][12][13]15,16,18,19]. Another issue is related to the differences in the image geometry acquired from these two devices [9,[11][12][13]15,16,18,19]. Thus, many co-registration approaches have been proposed (e.g., [9][10][11][12][13][14][15][16][17][18][19]), but, in general, these approaches might be divided into two main categories: area-and feature-based methods.…”

“…Another issue is related to the differences in the image geometry acquired from these two devices [9,[11][12][13]15,16,18,19]. Thus, many co-registration approaches have been proposed (e.g., [9][10][11][12][13][14][15][16][17][18][19]), but, in general, these approaches might be divided into two main categories: area-and feature-based methods. Nowadays, the Structure from Motion (SfM) methods, which are mostly based on the feature-based approach, are used for the co-registration of spaceborne SAR and optical images [9,[11][12][13]15,16,18,19].…”