Urban Flood Mapping With Bitemporal Multispectral Imagery Via a Self-Supervised Learning Framework

IEEE Geosci. Remote Sens. Mag.

Albrecht

Braham

et al. 2022

104

In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to, and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SSL in remote sensing and providing an extended study on data augmentations. Finally, we identify a list of promising directions of future research in SSL for earth observation (SSL4EO) to pave the way for fruitful interaction of both domains.

Section: A Generative Methodsmentioning

confidence: 99%

Self-Supervised Learning in Remote Sensing: A review

IEEE Geosci. Remote Sens. Mag.

Albrecht

Braham

et al. 2022

104

“…Notably, self-supervised learning figures out an effective fashion to fully exploit the unlabeled samples to excavate underly-ing category knowledge and learn useful features for some downstream tasks, such that the scarcity of labeled samples can be tackled to a certain extent. However, existing selfsupervised learning-based CD works [46][47][48] almost focus on multispectral/hyperspectral images [46] or cross-sensor images [47,48]. In [47], Chen and Bruzzone explored building a self-supervised pseudo-Siamese network for multispectral-SAR images CD based on contrastive learning.…”

Section: Related Workmentioning

confidence: 99%

Hybrid Variability Aware Network (HVANet): A Self-Supervised Deep Framework for Label-Free SAR Image Change Detection

Liu

2022

Remote Sensing

Synthetic aperture radar (SAR) image change detection (CD) aims to automatically recognize changes over the same geographic region by comparing prechange and postchange SAR images. However, the detection performance is usually subject to several restrictions and problems, including the absence of labeled SAR samples, inherent multiplicative speckle noise, and class imbalance. More importantly, for bitemporal SAR images, changed regions tend to present highly variable sizes, irregular shapes, and different textures, typically referred to as hybrid variabilities, further bringing great difficulties to CD. In this paper, we argue that these internal hybrid variabilities can also be used for learning stronger feature representation, and we propose a hybrid variability aware network (HVANet) for completely unsupervised label-free SAR image CD by taking inspiration from recent developments in deep self-supervised learning. First, since different changed regions may exhibit hybrid variabilities, it is necessary to enrich distinguishable information within the input features. To this end, in shallow feature extraction, we generalize the traditional spatial patch (SP) feature to allow for each pixel in bitemporal images to be represented at diverse scales and resolutions, called extended SP (ESP). Second, with the carefully customized ESP features, HVANet performs local spatial structure information extraction and multiscale–multiresolution (MS-MR) information encoding simultaneously through a local spatial stream and a scale-resolution stream, respectively. Intrinsically, HVANet projects the ESP features into a new high-level feature space, where the change identification becomes easier. Third, to train the framework effectively, a self-supervision layer is attached to the top of the HVANet to enable the two-stream feature learning and recognition of changed pixels in the corresponding feature space, in a self-supervised manner. Experimental results on three low/medium-resolution SAR datasets demonstrate the effectiveness and superiority of the proposed framework in unsupervised SAR CD tasks.

“…Meanwhile, Lu et al [41] utilized low-cost sparse point labels to extract water bodies through a neighbor feature aggregation network, although the weak annotations still cannot produce better results than dense ones in actual applications. Notably, Peng et al [20] designed a self-supervised learning framework based on an autoencoder to detect floodwater from patch-based bitemporal multispectral imagery. It is encouraging that decent flood mapping results could be achieved without requiring labeled data, despite the drawbacks such as the learning uncertainty from the autoencoder, the requirement of bitemporal imagery, and the coarser extraction results due to the patch-based training pipeline.…”

Section: Related Workmentioning

confidence: 99%

“…However, during the disaster period, optical satellite data may have poor quality due to cloud cover or not coincide with flood peaks because of low revisit frequency [19]. In light of the respective advantages of SAR and optical sensors, the integrated use of multitemporal [15], [20] and multimodal [19], [21] satellite data also gained attention recently, although some specific types of data may often be unavailable during floods [15] and image registration problems always exist [20]. Noteworthily, the aerial imagery from airplanes [12], [18] and, in recent years, unmanned aerial vehicles (UAVs) [2], [22] exhibited strengths in the aspects of very high spatial resolution and flexible acquisition time during disaster events [12], which can provide a critical profile of flooded areas insusceptible to cloud cover under complex This work is licensed under a Creative Commons Attribution 4.0 License.…”

mentioning

confidence: 99%

“…Furthermore, the majority of existing studies are focused on mapping floodwater based on low and medium spatial resolution imagery in rural regions with homogenous characteristics [15]. There is a lack of investigations using high spatial resolution data in urban areas with complex and diverse contexts [20]. For instance, in spite of the access to aerial imagery in a few real flooding cases [18], detecting floodwater in dark shadows is still challenging due to the difficulties of designing the handcrafted features of shadows based on conventional methods.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Enhancement of Urban Floodwater Mapping From Aerial Imagery With Dense Shadows via Semisupervised Learning

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Zhang

et al. 2022

Timely and accurate mapping of floodwater in urban areas from aerial imagery is critical to support emergency response and rescue work. However, massive shadows cast by buildings and trees over dense built-up urban areas can cause a significant underestimation of flood outcomes, and few studies for flood monitoring explore this in current state-of-the-art approaches. Meanwhile, recent deep learning (DL) algorithms have reported superior performance in flood mapping over conventional machine learning methods. Nevertheless, acquiring a large amount of training data remains challenging in the DL paradigm. In this study, to exploit the potential of the DL algorithm in detecting all visible (including shadowed and nonshadowed) floodwater with limited training samples from aerial imagery, we designed a modified fully convolutional network and combined it with a deep semisupervised learning framework integrating consistency regularization and RandMix strategy into the floodwater detection workflow. Besides, the testtime augmentation technique was leveraged to improve the model performance in the evaluation phase. Extensive experiments on the 2013 Calgary flood demonstrated the effectiveness of our approach on extracting visible floodwater in urban areas with dense shadows. Notably, our method could accurately detect more floodwater with a largely reduced number of labeled training samples, which is a considerable enhancement in the applicability and availability of DL algorithms for flood monitoring in densely shadowed urban areas.