Fast Cloud Segmentation Using Convolutional Neural Networks

Drönner, Johannes; Korfhage, Nikolaus; Egli, Sebastian; Mühling, Markus; Thies, Boris; Bendix, J.; Freisleben, Bernd; Seeger, Bernhard

doi:10.3390/rs10111782

Cited by 88 publications

(59 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One alternative was to employ an operational cloud database, such as in the work by Drönner et al [27], who used the well-validated Cloud Mask from the CLAAS-2 dataset [33]. However, that dataset was derived from geostationary Meteosat Spinning Enhanced Visible and Infrared Imager (SEVIRI) measurements for the time frame 2004-2015, which was not appropriate in training our DL model of cloud classification in terms of spatial, spectral, or temporal resolution.…”

Section: Training Datasetmentioning

confidence: 99%

“…Zhan et al [26] performed cloud classification tasks on Red-GreenBlue (RGB) images using FCN-based architecture. Drönner et al [27] further extended FCN-based architecture to an architecture that could identify RS multispectral data. So far, the downsampling technique (pooling or striding) has been adopted in most FCN-based architecture to extract features.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Clouds Classification from Sentinel-2 Imagery with Deep Residual Learning and Semantic Image Segmentation

et al. 2019

View full text Add to dashboard Cite

Detecting changes in land use and land cover (LULC) from space has long been the main goal of satellite remote sensing (RS), yet the existing and available algorithms for cloud classification are not reliable enough to attain this goal in an automated fashion. Clouds are very strong optical signals that dominate the results of change detection if they are not removed completely from imagery. As various architectures of deep learning (DL) have been proposed and advanced quickly, their potential in perceptual tasks has been widely accepted and successfully applied to many fields. A comprehensive survey of DL in RS has been reviewed, and the RS community has been suggested to be leading researchers in DL. Based on deep residual learning, semantic image segmentation, and the concept of atrous convolution, we propose a new DL architecture, named CloudNet, with an enhanced capability of feature extraction for classifying cloud and haze from Sentinel-2 imagery, with the intention of supporting automatic change detection in LULC. To ensure the quality of the training dataset, scene classification maps of Taiwan processed by Sen2cor were visually examined and edited, resulting in a total of 12,769 sub-images with a standard size of 224 × 224 pixels, cut from the Sen2cor-corrected images and compiled in a trainset. The data augmentation technique enabled CloudNet to have stable cirrus identification capability without extensive training data. Compared to the traditional method and other DL methods, CloudNet had higher accuracy in cloud and haze classification, as well as better performance in cirrus cloud recognition. CloudNet will be incorporated into the Open Access Satellite Image Service to facilitate change detection by using Sentinel-2 imagery on a regular and automatic basis.

show abstract

Section: Training Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Clouds Classification from Sentinel-2 Imagery with Deep Residual Learning and Semantic Image Segmentation

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Emerging new networks, such as U-Net [42] and DenseNet [43], have also been applied in remote sensing image semantic segmentation [44]. The application scenario also extends from surface geographic objects to continuous phenomena such as highly dynamic clouds [45]. Some studies introduce the attention mechanism [46,47] to achieve an ideal segmentation effect by suppressing low-level features and noise through high-level features.…”

Section: Cnn Seriesmentioning

confidence: 99%

Multi-Scale Semantic Segmentation and Spatial Relationship Recognition of Remote Sensing Images Based on an Attention Model

Cui

Wang

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

A comprehensive interpretation of remote sensing images involves not only remote sensing object recognition but also the recognition of spatial relations between objects. Especially in the case of different objects with the same spectrum, the spatial relationship can help interpret remote sensing objects more accurately. Compared with traditional remote sensing object recognition methods, deep learning has the advantages of high accuracy and strong generalizability regarding scene classification and semantic segmentation. However, it is difficult to simultaneously recognize remote sensing objects and their spatial relationship from end-to-end only relying on present deep learning networks. To address this problem, we propose a multi-scale remote sensing image interpretation network, called the MSRIN. The architecture of the MSRIN is a parallel deep neural network based on a fully convolutional network (FCN), a U-Net, and a long short-term memory network (LSTM). The MSRIN recognizes remote sensing objects and their spatial relationship through three processes. First, the MSRIN defines a multi-scale remote sensing image caption strategy and simultaneously segments the same image using the FCN and U-Net on different spatial scales so that a two-scale hierarchy is formed. The output of the FCN and U-Net are masked to obtain the location and boundaries of remote sensing objects. Second, using an attention-based LSTM, the remote sensing image captions include the remote sensing objects (nouns) and their spatial relationships described with natural language. Finally, we designed a remote sensing object recognition and correction mechanism to build the relationship between nouns in captions and object mask graphs using an attention weight matrix to transfer the spatial relationship from captions to objects mask graphs. In other words, the MSRIN simultaneously realizes the semantic segmentation of the remote sensing objects and their spatial relationship identification end-to-end. Experimental results demonstrated that the matching rate between samples and the mask graph increased by 67.37 percentage points, and the matching rate between nouns and the mask graph increased by 41.78 percentage points compared to before correction. The proposed MSRIN has achieved remarkable results. the existing studies do not address the interpretation of spatial relationships between remote sensing objects, which limits the understanding of remote sensing objects, especially when the phenomenon of different objects with the same spectrum in remote sensing appears.The phenomenon of different objects with the same spectrum in remote sensing is quite common. It is difficult to identify objects only by their own textures, spectra, and shape information. Object identification requires multi-scale semantic information and spatially adjacent objects to assist in decision-making. The spatial relationship between remote sensing objects is of great significance to the recognition of remote sensing objects when different objects have the same spectru...

show abstract

“…Cloud segmentation can be treated as an application of image segmentation, and therefore, applying semantic segmentation techniques for cloud detection is a reasonable consideration. Moreover, existing approaches based on deep learning for cloud segmentation have largely concentrated on satellite data (Drönner et al, ; Lu et al, ). Hence, it deserves to exploring the cloud segmentation performance by means of deep learning on the ground‐based cloud dataset.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Solution for Semantic Segmentation of Three Ground‐based Cloud Datasets

Song

Cui²,

Liu

2020

Earth and Space Science

View full text Add to dashboard Cite

The machine learning approach has shown its state-of-the-art ability to handle segmentation and detection tasks. It is increasingly employed to extract patterns and spatiotemporal features from the ever-increasing stream of Earth system data. However, there is still a significant challenge, which is the generalization capability of the model on cloud images in different types and weather conditions. After studying several popular methods, we propose a semantic segmentation neural network for cloud segmentation. It extracts features learned by source and target domains in an end-to-end behavior, which can address the problem of significant lack of labels in the observed cloud image data. It is further evaluated on the Singapore Whole Sky Image Segmentation (SWIMSEG) dataset by using Mean Intersection-over-Union, recall, F-score, and accuracy matrices. The scores of these matrices are 86%, 97%, 92%, and 96%, which prove that it has excellent efficiency and robustness. Most importantly, a new benchmark based on the SWIMSEG dataset for the task of cloud segmentation is introduced. The others, BENCHMARK, Cirrus Cumulus Stratus Nimbus are evaluated through the model trained from the SWIMSEG dataset by way of visualization. Plain Language Summary The machine learning approaches offer a new view about howto effectively and comprehensively understand ground-based cloud datasets. The essential advantage of deep learning methods is that it can extract more critical cloud features automatically than traditional algorithms, such as spatiotemporal features. Therefore, it is worth exploring the possibility of cloud segmentation with the help of deep learning techniques. We first introduce a semantic segmentation neural network for cloud segmentation problems after measuring a few classic neural networks. The results exceed the traditional methods by a large margin with standard evaluation matrices, such as Mean Intersection-over-Union, recall, F-score, and accuracy. The scores achieved here may accomplish as a baseline for competitive development. Second, the trained model is used to produce a few cloud masks in two public datasets: BENCHMARK, Singapore Whole Sky Image Segmentation, and their respective performance is further evaluated. Finally, the segmentation results show the excellent performance and generalization in another untrained dataset, Cirrus Cumulus Stratus Nimbus.

show abstract

Fast Cloud Segmentation Using Convolutional Neural Networks

Cited by 88 publications

References 47 publications

Clouds Classification from Sentinel-2 Imagery with Deep Residual Learning and Semantic Image Segmentation

Clouds Classification from Sentinel-2 Imagery with Deep Residual Learning and Semantic Image Segmentation

Multi-Scale Semantic Segmentation and Spatial Relationship Recognition of Remote Sensing Images Based on an Attention Model

An Efficient Solution for Semantic Segmentation of Three Ground‐based Cloud Datasets

Contact Info

Product

Resources

About