Automated Classification of Crop Types and Condition in a Mediterranean Area Using a Fine-Tuned Convolutional Neural Network

Abstract

Abstract. Crop classification based on satellite and aerial imagery is a recurrent application in remote sensing. It has been used as input for creating and updating agricultural inventories, yield prediction and land management. In the context of the Common Agricultural Policy (CAP), farmers get subsidies based on the crop area cultivated. The correspondence between the declared and the actual crop needs to be monitored every year, and the parcels must be properly maintained, without signs of abandonment. In this work, Sentinel- 2 time series images and 4-band Very High Resolution (VHR) aerial orthoimages from the Spanish National Programme of Aerial Orthophotography (PNOA) were combined in a pre-trained Convolutional Neural Network (CNN) (VGG-19) adapted with a double goal: (i) the classification of agricultural parcels in different crop types; and (ii) the identification of crop condition (i.e., abandoned vs. non-abandoned) of permanent crops in a Mediterranean area of Spain. A total of 1237 crop parcels from the CAP declarations of 2019 were used as ground truth to classify into cereals, fruit trees, olive trees, vineyards, grasslands and arable land, from which 80% were used for training and 20% for testing. The overall accuracy obtained was greater than 93% both, at parcel and area levels. Olive trees were the least accurate crop, mostly misclassified with fruit trees, and young vineyards were slightly confused with cereal and arable land. In the assessment of crop condition, only 9.65% of the abandoned plots were missed (omission errors), and 7.21% of plots were over-detected (commission errors), having a 99% of overall accuracy from a total of 1931 image subset samples. The proposed methodology based on CNN is promising for its operational application in crop monitoring and in the detection of abandonments in the context of CAP subsidies, but a more exhaustive number of training samples is needed for extension to other crop types and geographical areas.

Deep Learning Models for the Classification of Crops in Aerial Imagery: A Review

“…To improve their model's The majority of the studies considered in this review utilized DL techniques for crop classification employing remote sensing (RS) data, obtained from satellite or UAV imagery. Two studies, namely [17,18], employed aerial orthoimages of extremely high resolution obtained from aeroplanes, with the former utilizing Sentinel-2 and the latter moderate resolution imaging spectroradiometer (MODIS) satellite imagery. Additionally, reference [19,20] relied on spectral data derived from AVIRIS and ROSIS spectral sensors, respectively.…”

“…Table 3 presents a list of papers where researchers have employed models for crop classification using data from two sources, as opposed to just one. The papers [17][18][19][20] utilized data acquired from satellites and aircraft, while [65][66][67][68] utilized data from satellite and UAV systems. Among the different network versions implemented, convolutionalbased models are the most common.…”

“…Among the different network versions implemented, convolutionalbased models are the most common. The aim of the study described in paper [17] was to develop a CNN architecture that could accurately classify various crop types in agricultural parcels using both very high resolution (VHR) aerial imagery and Sentinel-2 time-series data. The CNN was trained on a combination of VHR and multispectral images and achieved an overall classification accuracy of over 93% for different crop types, such as cereals, fruit trees, olive trees, vineyards, grasslands, and arable land.…”

Assessing the Sentinel-2 Capabilities to Identify Abandoned Crops Using Deep Learning

“…However, machine learning and deep learning methods have become state of the art for this task. While the first require feeding the algorithms with meaningful features, deep learning architectures excel in extracting these features encoded in the data [9], and usually achieve very high accuracies in crop identification tasks [10,11], and also in abandonment detection [12]. Despite their excellent results, the majority of deep learning models are considered as "black-boxes", whereby another relevant trend is seeking interpretability of these algorithms, for improved decision making [13].…”

Deep learning para la clasificación de usos de suelo agrícola con Sentinel-2

“…La comunidad científica lleva caracterizando usos del suelo desde hace décadas utilizando datos de sensores como el Moderate Resolution Imaging Spectroradiometer (MODIS) a bordo de los satélites Terra y Aqua con resoluciones espaciales desde los 250 m (Zhan et al, 2000) a 1 km (Friedl et al, 2002), y sensores de más alta resolución espacial (30 m) como el Thematic Mapper (TM) y el Operational Land Imager (OLI) a bordo de los satélites Landsat-5/7 (Zhu y Woodcock, 2014) y Landsat-8 respectivamente (Kussul et al, 2017). Gracias a la disponibilidad de datos del programa Copernicus de la Agencia Espacial Europea, han aparecido multitud de estudios de clasificación de usos del suelo utilizando datos del sensor MultiSpectral Imager (MSI) a bordo del satélite Sentinel-2 (Immitzer et al, 2016;Vuolo et al, 2018;Griffiths et al, 2019;Campos-Taberner et al, 2019a,b, González-Guerrero y Pons, 2020Ruiz et al, 2020).…”