Sen12ms – A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion

Abstract

This is a pre-print of a paper accepted for publication in the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Please refer to the original (open access) publication from September 2019. The availability of curated large-scale training data is a crucial factor for the development of well-generalizing deep learning methods for the extraction of geoinformation from multi-sensor remote sensing imagery. While quite some datasets have already been published by the community, most of them suffer from rather strong limitations, e.g. regarding spatial coverage, diversity or simply number of available samples. Exploiting the freely available data acquired by the Sentinel satellites of the Copernicus program implemented by the European Space Agency, as well as the cloud computing facilities of Google Earth Engine, we provide a dataset consisting of 180,662 triplets of dual-pol synthetic aperture radar (SAR) image patches, multi-spectral Sentinel-2 image patches, and MODIS land cover maps. With all patches being fully georeferenced at a 10 m ground sampling distance and covering all inhabited continents during all meteorological seasons, we expect the dataset to support the community in developing sophisticated deep learning-based approaches for common tasks such as scene classification or semantic segmentation for land cover mapping.

Generating Artificial Near Infrared Spectral Band From RGB Image Using Conditional Generative Adversarial Network

“…We use the multispectral images from SEN12MS dataset based on Sentinel-1 and Sentinel-2 dataset (Schmitt et al, 2019). The Sentinel-2 data from SEN12MS is level 1-C Top of Atmosphere reflectance (TOA) product.…”

Multisensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery

“…To conduct our experiments, we gather a novel large-scale data set called SEN12MS-CR for cloud removal. For this purpose, we build upon the openly available SEN12MS data set [27] of globally sampled coregistered S1 plus cloud-free S2 patches and complement the data set with coregistered [18] cloudy images close in time to the original observations. SEN12MS-CR consists of 169 nonoverlapping ROIs evenly distributed over all continents and meteorological seasons.…”

Weakly Supervised Semantic Segmentation of Satellite Images for Land Cover Mapping – Challenges and Opportunities

“…The SEN12MS dataset (Schmitt et al, 2019) was published in 2019 as the largest curated dataset dedicated to deep learning in remote sensing at that time. It consists of 180,662 patch triplets sampled over all meteorological seasons and all inhabited continents in order to represent a global distribution.…”

“…In this paper, we discuss the problem of weakly supervised learning of models for land cover prediction from satellite data. For this purpose, we focus on the freely available global imagery provided by the Sentinel-1 and Sentinel-2 missions of the European Copernicus program (Torres et al, 2012, Drusch et al, 2012) and a simplified version of the land cover classification scheme of the International Geosphere-Biosphere Programme (IGBP) (Loveland, Belward, 1997), which is reflected by the SEN12MS dataset (Schmitt et al, 2019) and the 2020 IEEE-GRSS Data Fusion Contest (DFC2020) (Yokoya et al, 2020). Besides a description of the challenge and how SEN12MS and DFC2020 are addressing it, baseline results using off-the-shelf deep learning models are provided to highlight the importance of dedicated research.…”