Mapping Crop Types in Southeast India with Smartphone Crowdsourcing and Deep Learning

Abstract

High resolution satellite imagery and modern machine learning methods hold the potential to fill existing data gaps in where crops are grown around the world at a sub-field level. However, high resolution crop type maps have remained challenging to create in developing regions due to a lack of ground truth labels for model development. In this work, we explore the use of crowdsourced data, Sentinel-2 and DigitalGlobe imagery, and convolutional neural networks (CNNs) for crop type mapping in India. Plantix, a free app that uses image recognition to help farmers diagnose crop diseases, logged 9 million geolocated photos from 2017–2019 in India, 2 million of which are in the states of Andhra Pradesh and Telangana in India. Crop type labels based on farmer-submitted images were added by domain experts and deep CNNs. The resulting dataset of crop type at coordinates is high in volume, but also high in noise due to location inaccuracies, submissions from out-of-field, and labeling errors. We employed a number of steps to clean the dataset, which included training a CNN on very high resolution DigitalGlobe imagery to filter for points that are within a crop field. With this cleaned dataset, we extracted Sentinel time series at each point and trained another CNN to predict the crop type at each pixel. When evaluated on the highest quality subset of crowdsourced data, the CNN distinguishes rice, cotton, and “other” crops with 74% accuracy in a 3-way classification and outperforms a random forest trained on harmonic regression features. Furthermore, model performance remains stable when low quality points are introduced into the training set. Our results illustrate the potential of non-traditional, high-volume/high-noise datasets for crop type mapping, some improvements that neural networks can achieve over random forests, and the robustness of such methods against moderate levels of training set noise. Lastly, we caution that obstacles like the lack of good Sentinel-2 cloud mask, imperfect mobile device location accuracy, and preservation of privacy while improving data access will need to be addressed before crowdsourcing can widely and reliably be used to map crops in smallholder systems.

Crop Type and Land Cover Mapping in Northern Malawi Using the Integration of Sentinel-1, Sentinel-2, and PlanetScope Satellite Data

“…A combination of field ground truth data and samples from Google Earth Pro and DigitalGlobe images, as well as ancillary information, was used to train and validate the random forest algorithm to attain more than 85% accuracy in the experiment (v) ( Table 4). The accuracy attained suggests that using reference data from diverse sources can improve classification results, as observed in other studies [23,74]. This observation implies that the level of accuracy obtained in our experiments can be further improved with more training and validation samples.…”

“…In future, crowdsourcing should be used to collect more field samples by training farmers to use smartphones and other spatial data collection applications and tools to geolocate and record farm and ancillary information for training and validating classifiers. Using crowdsourcing to collect crop type samples has been experimented with in other contexts and found to have contributed to getting large volumes of training data for mapping crop types more accurately [23]. Adding more training samples will also allow classification algorithms to identify the specific crops in intercropped fields.…”

“…The relatively higher 10-m resolution produces improved crop and land cover classification accuracy. Several previous studies have shown that Sentinel-2 data can be used to identify crop types [21,23,41].…”

“…In resource-poor settings in SSA where climate change is predicted to have more severe impacts, the advancements in satellite remote sensing and algorithms present an opportunity to develop crop type maps as decision support and resilience-building to food insecurity. Wang et al [23] contend that remote sensing can aid with crop type mapping if field-level ground data are collected using field surveys to train and validate models to help identify crop types. Even with detailed ground truth data, mapping crop types in the context of smallholder agriculture remains complex because farmlands are smaller, farm-level species are more diverse, agricultural practices are more variable, and intercropping and crop rotations are predominant [8,24,25].…”

“…Previous studies have applied such an integrative approach to improving classification accuracies. For instance, Wang et al [23] used crowdsourced ground truth data with Sentinel-1 and Sentinel-2 data and applied a deep learning classification technique to identify rice and cotton crops in India. Wang, Azzari, and Lobell [26] used Landsat data to map crop types in the US Midwest.…”