Object-Based Multi-Temporal and Multi-Source Land Cover Mapping Leveraging Hierarchical Class Relationships

Gbodjo, Yawogan Jean Eudes; Ienco, Dino; Leroux, Louise; Interdonato, Roberto; Gaetano, Raffaele; Ndao, Babacar

doi:10.3390/rs12172814

Cited by 30 publications

(17 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result shows how integrating privileged information at the training stage is beneficial for the classification process, even if such information is not available when performing predictions. In addition, the reported results are coherent with the results of (Gbodjo et al, 2019) in which the MLP approach exhibits competitive performances w.r.t. standard machine learning approaches usually employed to deal with land use land cover mapping from SITS data.…”

Section: General Behaviorsupporting

confidence: 84%

See 1 more Smart Citation

Generalized Knowledge Distillation for Multi-Sensor Remote Sensing Classification: An Application to Land Cover Mapping

Ienco

Gbodjo

Gaetano

et al. 2020

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Due to the proliferation of Earth Observation programmes, information at different spatial, spectral and temporal resolution is collected by means of various sensors (optical, radar, hyperspectral, LiDAR, etc.). Despite such abundance of information, it is not always possible to obtain a complete coverage of the same area (especially for large ones) from all the different sensors due to: (i) atmospheric conditions and/or (ii) acquisition cost. In this context of data (or modalities) misalignment, only part of the area under consideration could be covered by the different sensors (modalities). Unfortunately, standard machine learning approaches commonly employed in operational Earth monitoring systems require consistency between training and test data (i.e., they need to match the same information schema). Such a constraint limits the use of additional fruitful information, i.e., information coming from a particular sensor that may be available at training but not at test time. Recently, a framework able to manage such information misalignment between training and test information is proposed under the name of Generalized Knowledge Distillation (GKD). With the aim to provide a proof of concept of GKD in the context of multi-source Earth Observation analysis, here we provide a Generalized Knowledge Distillation framework for land use land cover mapping involving radar (Sentinel-1) and optical (Sentinel-2) satellite image time series data (SITS). Considering that part of the optical information may not be available due to bad atmospheric conditions, we make the assumption that radar SITS are always available (at both training and test time) while optical SITS are only accessible when the model is learnt (i.e., it is considered as privileged information). Evaluations are carried out on a real-world study area in the southwest of France, namely Dordogne, considering a mapping task involving seven different land use land cover classes. Experimental results underline how the additional (privileged) information ameliorates the results of the radar based classification with a main gain on the agricultural classes.

show abstract

Section: General Behaviorsupporting

confidence: 84%

“…For our student model, we use a simple Multilayer Perceptron (MLP). The MLP network has recently demonstrated its ability to deal with object-based SITS classification (Gbodjo et al, 2019). The MLP architecture we employ is summarized in…”

Section: Student Modelmentioning

confidence: 99%

Generalized Knowledge Distillation for Multi-Sensor Remote Sensing Classification: An Application to Land Cover Mapping

Ienco

Gbodjo

Gaetano

et al. 2020

ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

show abstract

“…• Hierarchical object based RNN (Hob2srnn) proposed in [33]. Such a method is an extension of the Recurrent Neural Network architecture, equipped with an attention mechanism, developed in the context of object-based satellite image time series classification.…”

Section: Methodsmentioning

confidence: 99%

Attentive Spatial Temporal Graph CNN for Land Cover Mapping From Multi Temporal Remote Sensing Data

et al. 2021

Self Cite

View full text Add to dashboard Cite

Satellite image time series (SITS) collected by modern Earth Observation (EO) systems represent a valuable source of information that supports several tasks related to the monitoring of the Earth surface dynamics over large areas. A main challenge is then to design methods able to leverage the complementarity between the temporal dynamics and the spatial patterns that characterize these data structures. Focusing on land cover classification (or mapping) tasks, the majority of approaches dealing with SITS data only considers the temporal dimension, while the integration of the spatial context is frequently neglected. In this work, we propose an attentive spatial temporal graph convolutional neural network that exploits both spatial and temporal dimensions in SITS. Despite the fact that this neural network model is well suited to deal with spatio-temporal information, this is the first work that considers it for the analysis of SITS data. Experiments are conducted on two study areas characterized by different land cover landscapes and real-world operational constraints (i.e., limited labeled data due to acquisition costs). The results show that our model consistently outperforms all the competing methods obtaining a performance gain, in terms of F-Measure, of at least 5 points with respect to the best competing approaches on both benchmarks. INDEX TERMSSpatial temporal graph convolutional neural network, attention-based neural network, object-based image classification, satellite image time series, land cover classification, deep learning.

show abstract

“…However, the majority of land cover maps are still only relying on spectral information or as in recent studies, in spectral and spatial information. Consequently, the use of temporal dependencies has been poorly investigated as explained in (Gómez et al, 2016) and (Gbodjo et al, 2020).…”

Section: Contextmentioning

confidence: 99%

Benchmarking of Convolutional Neural Network Approaches for Vegetation Land Cover Mapping

Carpentier

Massé

Lavergne

et al. 2021

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Abstract. Satellite Image Time Series (SITS) are becoming available at high spatial, spectral and temporal resolutions across the globe by the latest remote sensing sensors. These series of images can be highly valuable when exploited by classification systems to produce frequently updated and accurate land cover maps. The richness of spectral, spatial and temporal features in SITS is a promising source of data for developing better classification algorithms. However, machine learning methods such as Random Forests (RF), despite their fruitful application to SITS to produce land cover maps, are structurally unable to properly handle intertwined spatial, spectral and temporal dynamics without breaking the structure of the data. Therefore, the present work proposes a comparative study of various deep learning algorithms from the Convolutional Neural Network (CNN) family and evaluate their performance on SITS classification. They are compared to the processing chain coined iota2, developed by the CESBIO and based on a RF model. Experiments are carried out in an operational context using with sparse annotations from 290 labeled polygons. Less than 80 000 pixel time series belonging to 8 land cover classes from a year of Sentinel-2 monthly syntheses are used. Results show on a test set of 131 polygons that CNNs using 3D convolutions in space and time are more accurate than 1D temporal, stacked 2D and RF approaches. Best-performing models are CNNs using spatio-temporal features, namely 3D-CNN, 2D-CNN and SpatioTempCNN, a two-stream model using both 1D and 3D convolutions.

show abstract

Object-Based Multi-Temporal and Multi-Source Land Cover Mapping Leveraging Hierarchical Class Relationships

Cited by 30 publications

References 48 publications

Generalized Knowledge Distillation for Multi-Sensor Remote Sensing Classification: An Application to Land Cover Mapping

Generalized Knowledge Distillation for Multi-Sensor Remote Sensing Classification: An Application to Land Cover Mapping

Attentive Spatial Temporal Graph CNN for Land Cover Mapping From Multi Temporal Remote Sensing Data

Benchmarking of Convolutional Neural Network Approaches for Vegetation Land Cover Mapping

Contact Info

Product

Resources

About