Temporal Convolutional Neural Network for the Classification of Satellite Image Time Series

Abstract

New remote sensing sensors now acquire high spatial and spectral Satellite Image Time Series (SITS) of the world. These series of images are a key component of classification systems that aim at obtaining up-to-date and accurate land cover maps of the Earth's surfaces. More specifically, the combination of the temporal, spectral and spatial resolutions of new SITS makes possible to monitor vegetation dynamics. Although traditional classification algorithms, such as Random Forest (RF), have been successfully applied for SITS classification, these algorithms do not make the most of the temporal domain. Conversely, some approaches that take into account the temporal dimension have recently been tested, especially Recurrent Neural Networks (RNNs). This paper proposes an exhaustive study of another deep learning approaches, namely Temporal Convolutional Neural Networks (TempCNNs) where convolutions are applied in the temporal dimension. The goal is to quantitatively and qualitatively evaluate the contribution of TempCNNs for SITS classification. This paper proposes a set of experiments performed on one million time series extracted from 46 Formosat-2 images. The experimental results show that TempCNNs are more accurate than RF and RNNs, that are the current state of the art for SITS classification. We also highlight some differences with results obtained in computer vision, e.g. about pooling layers. Moreover, we provide some general guidelines on the network architecture, common regularization mechanisms, and hyper-parameter values such as batch size. Finally, we assess the visual quality of the land cover maps produced by TempCNNs.

Data Acquisition and Processing for GeoAI Models to Support Sustainable Agricultural Practices

“…Their results, based on an architecture that includes one-dimension convolution and an inception module, outperformed traditional algorithms for land use classification including XGBoost, Random Forest, Support Vector Machine and recurrent deep neural networks. Pelletier et al [43] proposed a temporal convolutional neural network constructed with three convolutional layers, a dense layer and finally, a Softmax layer. Different to [42], authors of this study used three spectral bands of the available satellite imagery.…”

TS-CHIEF: a scalable and accurate forest algorithm for time series classification

“…Creating such a map requires observing the evolution of each of these quadrillions of geographic areas over a given period (e.g., one year) for which a set of time series is extracted to represent the evolution of the observed variables over time. These quadrillions of time series can each then be classified into land cover categories such as 'corn crop', 'eucalyptus forest' or 'urban' [36].…”

Towards a Multi-Temporal Deep Learning Approach for Mapping Urban Fabric Using Sentinel 2 Images

“…Conversely, recurrent neural networks, and particularly LSTMs, can process such data. However, they are not adequate for our purposes, giving that land use mapping aims at producing one classification map per series [28]. Various studies investigated the use of CNNs for processing multi-temporal data.…”

“…Nogueira et al [31] implemented a multi-branch CNN for vegetation mapping and confirmed its superiority to a traditional CNN operating on a temporal stack of images. Pelletier et al [28] proposed a temporal CNN for crop classification where convolutions are applied in the temporal domain. The literature review reveals that few studies investigated the temporal dimension of high-resolution satellite imagery such as Sentinel-2 (10 m) for mapping UF.…”