SegCloud: a novel cloud image segmentation model using deep Convolutional Neural Network for ground-based all-sky-view camera observation

Abstract

Abstract. Cloud detection and cloud properties have significant applications in weather forecast, signal attenuation analysis, and other cloud-related fields. Cloud image segmentation is the fundamental and important step to derive cloud cover. However, traditional segmentation methods rely on low-level visual features of clouds, and often fail to achieve satisfactory performance. Deep Convolutional Neural Networks (CNNs) are able to extract high-level feature information of object and have become the dominant methods in many image segmentation fields. Inspired by that, a novel deep CNN model named SegCloud is proposed and applied to accurate cloud segmentation based on ground-based observation. Architecturally, SegCloud possesses symmetric encoder-decoder structure. The encoder network combines low-level cloud features to form high-level cloud feature maps with low resolution, and the decoder network restores the obtained high-level cloud feature maps to the same resolution of input images. The softmax classifier finally achieves pixel-wise classification and outputs segmentation results. SegCloud has powerful cloud discrimination ability and can automatically segment the whole sky images obtained by a ground-based all-sky-view camera. Furthermore, a new database, which includes 400 whole sky images and manual-marked labels, is built to train and test the SegCloud model. The performance of SegCloud is validated by extensive experiments, which show that SegCloud is effective and accurate for ground-based cloud segmentation and achieves better results than traditional methods. Moreover, the accuracy and practicability of SegCloud is further proved by applying it to cloud cover estimation.

Data-Driven Cloud Clustering via a Rotationally Invariant Autoencoder

“…Early work on machine-based cloud classification algorithms integrated simple statistics with machine learning algorithms [4]- [7], and combined textual and cloud physical features [8], [9]. More recent work [10], [11], [13], [27], [28] takes advantage of state-of-the-art deep learning technologies reinforced by increasingly powerful modern computing hardware to achieve high classification accuracy in extracting relative features from images in cloud classification.…”

Using Gans to Augment Data for Cloud Image Segmentation Task

“…Send correspondence to S. Dev: soumyabrata.dev@ucd.ie a deep convolutional neural network (CloudSegNet) and reported a maximum F-score of nearly 0.90 as compared to less than 0.80 for previous efforts without deep networks [3,7]. Similarly, Xie et al [8] reported a pixel-wise classification accuracy of more than 95% after training their proposed architecture called SegCloud.…”

Applying self-supervised learning for semantic cloud segmentation of all-sky images

“…There are also studies on supervised learning techniques for classifying pixels in ASIs such as neural networks, support vector machines (SVMs), random forests and Bayesian classifiers (Taravat et al, 2014;Cheng and Lin, 2017;Ye et al, 2019). Lately, also deep learning approaches using convolutional neural networks (CNNs) were presented (Dev et al, 2019;Xie et al, 2020;Song et al, 2020). Although, they were trained in a purely supervised manner using relatively small datasets, the results outperform threshold-based state-of-the-art methods significantly.…”