SIPSA-Net: Shift-Invariant Pan Sharpening with Moving Object Alignment for Satellite Imagery

Lee, Jaehyup; Seo, Soomin; Kim, Munchurl

doi:10.1109/cvpr46437.2021.01003

Cited by 27 publications

(7 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2(b), due to the different sensors' capturing time, the clock skew between two sensors (e.g. visible and infrared) can lead to pixelmisalignment of image pairs, especially for locally moving objects such as cars on a highway [10]. As a result, clock skew causes the position and angle of the same object to be inconsistent in different modalities, see Fig.…”

Section: Discussionmentioning

confidence: 99%

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Yuan¹,

Wang²,

Wei³

2022

Preprint

View full text Add to dashboard Cite

Integrating multispectral data in object detection, especially visible and infrared images, has received great attention in recent years. Since visible (RGB) and infrared (IR) images can provide complementary information to handle light variations, the paired images are used in many fields, such as multispectral pedestrian detection, RGB-IR crowd counting and RGB-IR salient object detection. Compared with natural RGB-IR images, we find detection in aerial RGB-IR images suffers from cross-modal weakly misalignment problems, which are manifested in the position, size and angle deviations of the same object. In this paper, we mainly address the challenge of cross-modal weakly misalignment in aerial RGB-IR images. Specifically, we firstly explain and analyze the cause of the weakly misalignment problem. Then, we propose a Translation-Scale-Rotation Alignment (TSRA) module to address the problem by calibrating the feature maps from these two modalities. The module predicts the deviation between two modality objects through an alignment process and utilizes Modality-Selection (MS) strategy to improve the performance of alignment. Finally, a two-stream feature alignment detector (TSFADet) based on the TSRA module is constructed for RGB-IR object detection in aerial images. With comprehensive experiments on the public DroneVehicle datasets, we verify that our method reduces the effect of the cross-modal misalignment and achieve robust detection results.

show abstract

Section: Discussionmentioning

confidence: 99%

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Yuan¹,

Wang²,

Wei³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…These works consider geometric alignment models from in-plane translation only [18,19,20] to rotation and projective homography [9]. But while successfully generating high resolution hyperspectral images, they are only robust to very slight misalignment of the input image pairs of similar viewpoint and FoV, i.e.…”

Section: Hyperspectral Image Alignment Methodsmentioning

confidence: 99%

Direct Alignment Of Narrow Field-Of-View Hyperspectral Data And Full-View Rgb Image

Caron

Kessy

Mukaigawa

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

The novelty of this paper is the alignment method of narrow field-of-view hyperspectral images to full-view RGB images. The interest is to locate hyperspectral measurements in an environment described by an equirectangular image. But the very different modalities (3 vs. hundreds of channels) and fields-of-view are challenges for accurate alignment. We solve these problems within a dense direct alignment framework that optimizes the warping parameters together with those of a global illumination difference model. Our alignment code is shared with an example dataset available at github.com/jrl-umi3218/hsrgbalign.

show abstract

“…Lee J., et al [6] developed a shift-invariant pan-sharpening with moving object alignment (SIPSA-Net) for pan-sharpening. Here, a feature alignment module was employed for incorporating multiple features with respect to the MS and PAN domains.…”

Section: Other Pan Sharpening Techniquesmentioning

confidence: 99%

Study and Analysis of Various Pan-Sharpening Techniques: A Challenging Overview

Singh¹

2022

View full text Add to dashboard Cite

Satellite sensors acquire massive data to exploit and address numerous human tasks. Due to the physical constraints, the usage of high spatial resolution multispectral (MS) images cannot be acquired by the unique devices. Hence, it is necessary to develop techniques by incorporating the images of MS and the panchromatic (PAN), also called Pan sharpening in remote sensing applications. Hence, this survey analyzes 30 research existing papers in Pan sharpening, which is based on various techniques, like neural network-based techniques, image fusion-based techniques, guided filterbased techniques, and optimization-based techniques. The analysis is based on the publication year of the research paper, research techniques employed, performance evaluation measures, and achievement of the research techniques. Also, the issues in the approaches are explained. Moreover, the future scope is to develop an efficient Pan sharpening technique based on the limitations recognized from the existing research Pan sharpening techniques.

show abstract

SIPSA-Net: Shift-Invariant Pan Sharpening with Moving Object Alignment for Satellite Imagery

Cited by 27 publications

References 22 publications

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Translation, Scale and Rotation: Cross-Modal Alignment Meets RGB-Infrared Vehicle Detection

Direct Alignment Of Narrow Field-Of-View Hyperspectral Data And Full-View Rgb Image

Study and Analysis of Various Pan-Sharpening Techniques: A Challenging Overview

Contact Info

Product

Resources

About