CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images

Kang, Wenchao; Xiang, Yuming; Wang, Feng; You, Hongjian

doi:10.1109/jstars.2022.3144587

Cited by 32 publications

(23 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considering the inputs use the same modality, weight sharing ensures that the network produces similar feature representation for the two inputs. However, optical and SAR inputs show different characteristics that need to be processed using different networks [29]. Furthermore, in our case, pre-change input consists of two different modalities (optical and SAR), while the post-change input contains only SAR data.…”

Section: Introductionmentioning

confidence: 99%

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Saha

Shahzad

Ebel

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Change detection using satellite/aerial images is used to quantify the impacts of many natural and man-made disasters. At the occurrence of such events, both pre-change optical and Synthetic Aperture Radar (SAR) images can be obtained by going back in time. However, the availability of the post-change optical image is often hindered by the presence of artifacts like clouds.To circumnavigate this, we propose a novel change detection data setting which uses both optical and SAR images prechange, yet only SAR imagery post-change. For this challenging scenario, we propose a Siamese network that processes the pre-change and post-change SAR inputs using a shared set of weights, while the pre-change optical input is processed using a network that do not share the weights with the SAR inputs. The encoded weights from the three networks are fused and finally decoded using a common decoder to obtain the change map. Our model effectively fuses multi-sensor information and can obtain satisfactory result despite the absence of the post-change optical image. Experimental results on a multi-sensor urban dataset demonstrate the effectiveness of the proposed approach. Code is available at https:

show abstract

Section: Introductionmentioning

confidence: 99%

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Saha

Shahzad

Ebel

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…To further demonstrate the effectiveness of our fusion method, we compare it with other fusion strategies that are widely used in RGB-Depth and RGB-Thermal perception tasks, including: channel-wise weighted feature fusion (CWF) [14], cross gates (CRGs) [15], cross reference module (CRM) [16], gated information fusion (GIF) [17], and a fusion method for PAN and MS data fusion, i.e., the adaptive feature fusion module (AFFM) [18]. For CWF, CRGs, and GIF, we reimplement them in strict accordance with the paper; for SCA and AFMM, we use the codes the authors provided.…”

Section: F Overall Resultsmentioning

confidence: 99%

“…CRGs [15] generates channel weights for PAN and MS modalities, respectively, and then applies them crosswise, as shown in Fig. 7(b).…”

Section: F Overall Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Building Detection From Panchromatic and Multispectral Images With Dual-Stream Asymmetric Fusion Networks

Huang

Liu

Zhou

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Building detection from panchromatic (PAN) and Multi-spectral (MS) images is an essential task for many practical applications. In this paper, a dual-stream asymmetric fusion network is proposed, named DAFNet. DAFNet can achieve effective information fusion at the feature level. It obtains better building detection performance from three perspectives: 1) A two-stream network structure is designed to guarantee the ability to extract information from PAN and MS images; 2) An asymmetric feature fusion (AFF) module is proposed to fuse features efficiently and concisely; 3) Two consistency regularization losses, i.e., PAN information preservation (PiP) loss and cross-modal semantic consistency (CSC) loss are applied to further explore the consistency between features for better fusion. The experiments are conducted on a challenging building detection dataset collected from GaoFen-2 satellite images. Comprehensive evaluations on 12 popular detection methods demonstrate the superiority of our DAFNet compared with the existing state-of-the-art fusion methods. We reveal that feature-level fusion is more suitable for building detection from PAN-MS images. Code is released at https://github.com/floatingstarZ/DAFNet

show abstract

“…[2], [3] and semantic segmentation [4], [5]. While clouds are characterised in great detail [6], [7] and different approaches for handling them have been investigated, less effort has been spent to investigate what exactly its effects on remote sensing applications are.…”

Section: Introductionmentioning

confidence: 99%

Explaining the Effects of Clouds on Remote Sensing Scene Classification

Gawlikowski

Ebel

Schmitt

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Most of Earth is covered by haze or clouds, impeding the constant monitoring of our planet. Preceding works have documented the detrimental effects of cloud coverage on remote sensing applications and proposed ways to approach this issue. However, up-to-now, little effort has been spent on understanding how exactly atmospheric disturbances impede the application of modern machine learning methods to Earth observation data. Specifically, we consider the effects of haze and cloud coverage on a scene classification task. We provide a thorough investigation of how classifiers trained on cloud-free data fail once they encounter noisy imagery -a common scenario encountered when deploying pretrained models for remote sensing to real use cases. We show how and why remote sensing scene classification suffers from cloud coverage. Based on a multi-stage analysis, including explainability approaches applied to the predictions, we work out four different types of effects that clouds have on the scene prediction. The contribution of our work is to deepen the understanding of the effects of clouds on common remote sensing applications, and consequently guide the development of more robust methods.

show abstract

CFNet: A Cross Fusion Network for Joint Land Cover Classification Using Optical and SAR Images

Cited by 32 publications

References 37 publications

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Supervised Change Detection Using Prechange Optical-SAR and Postchange SAR Data

Building Detection From Panchromatic and Multispectral Images With Dual-Stream Asymmetric Fusion Networks

Explaining the Effects of Clouds on Remote Sensing Scene Classification

Contact Info

Product

Resources

About