Multilevel Feature Fusion-Based CNN for Local Climate Zone Classification From Sentinel-2 Images: Benchmark Results on the So2Sat LCZ42 Dataset

Abstract

As a unique classification scheme for urban forms and functions, the local climate zone (LCZ) system provides essential general information for any studies related to urban environments, especially on a large scale. Remote sensing data-based classification approaches are the key to large-scale mapping and monitoring of LCZs. The potential of deep learning-based approaches is not yet fully explored, even though advanced convolutional neural networks (CNNs) continue to push the frontiers for various computer vision tasks. One reason is that published studies are based on different datasets, usually at a regional scale, which makes it impossible to fairly and consistently compare the potential of different CNNs for real-world scenarios. This article is based on the big So2Sat LCZ42 benchmark dataset dedicated to LCZ classification. Using this dataset, we studied a range of CNNs of varying sizes. In addition, we proposed a CNN to classify LCZs from Sentinel-2 images, Sen2LCZ-Net. Using this base network, we propose fusing multilevel features using the extended Sen2LCZ-Net-MF. With this proposed simple network architecture, and the highly competitive benchmark dataset, we obtain results that are better than those obtained by the state-of-the-art CNNs, while requiring less computation with fewer layers and parameters. Large-scale LCZ classification examples of completely unseen areas are presented, demonstrating the potential of our proposed Sen2LCZ-Net-MF

LCZ Generator: A Web Application to Create Local Climate Zone Maps

“…In case updates occur in the future, they will be tracked via the software version number and described in the changelog available on the Github Issue page. For example, some successfully tested the use of object-based image analysis (Collins and Dronova, 2019;Simanjuntak et al, 2019), others obtained promising results using (residual) convolutional neural networks (Qiu et al, 2019(Qiu et al, , 2020Yoo et al, 2019;Liu and Shi, 2020;Rosentreter et al, 2020;Zhu et al, 2020). Yet to date, the feasibility of such procedures for large-scale LCZ mapping has not yet been demonstrated (Demuzere et al, 2020a).…”

Mapping Local Climate Zones: A Bibliometric Meta-Analysis and Systematic Review

“…Liu & Shi (2020) tested scene sizes ranging from 10×10 to 96×96 pixels, and found that scene sizes from 32×32 to 64×64 pixels are suitable. Similarily, some studies set the scene size to 32×32 pixels for LCZ classification (Feng et al, 2019;Qiu et al, 2020c;Rosentreter et al, 2020).…”

“…This LCZ sample set is labeled by a group of domain experts following a carefully designed workflow and evaluation process, which has an overall confidence level of 85%. The So2Sat LCZ42 data set has been used for LCZ classification in several cities (Feng et al, 2019;Jing et al, 2019;Qiu et al, 2018cQiu et al, , 2020cTaubenböck et al, 2020;Yang et al, 2019b (Yokoya et al, 2018). The top four teams used a variety of classification algorithms deriving from computer vision and machine learning (dos Anjos et al, 2017;Sukhanov et al, 2017;Xu et al, 2017a;Yokoya et al, 2017).…”

A Parcel-Based Deep-Learning Classification to Map Local Climate Zones From Sentinel-2 Images

“…2(a)] [18]. For example, a benchmark dataset, So2Sat LCZ42, contains a large number of image patches of 320 × 320 m 2 size collected from Sentinel-2 images, which has been widely used to train supervised classifiers for LCZ mapping [6], [19].…”

“…This proposed method was trained with the So2Sat LCZ42 dataset and tested in four sites. The results were compared with those from classifying regular image patches using a sliding-window approach [19] to demonstrate the improvement of the proposed method.…”