AirSurf-Lettuce: an aerial image analysis platform for ultra-scale field phenotyping and precision agriculture using computer vision and deep learning

Abstract

Aerial imagery is regularly used by farmers and growers to monitor crops during the growing season. To extract meaningful phenotypic information from large-scale aerial images collected regularly from the field, high-throughput analytic solutions are required, which not only produce high-quality measures of key crop traits, but also support agricultural practitioners to make reliable management decisions of their crops. Here, we report AirSurf-Lettuce, an automated and open-source aerial image analysis platform that combines modern computer vision, up-to-date machine learning, and modular software engineering to measure yield-related phenotypes of millions of lettuces across the field. Utilising ultra-large normalized difference vegetation index (NDVI) images acquired by fixed-wing light aircrafts together with a deep-learning classifier trained with over 100,000 labelled lettuce signals, the platform is capable of scoring and categorising iceberg lettuces with high accuracy (>98%). Furthermore, novel analysis functions have been developed to map lettuce size distribution in the field, based on which global positioning system (GPS) tagged harvest regions can be derived to enable growers and farmers’ precise harvest strategies and marketability estimates before the harvest.

GranoScan: an AI-powered mobile app for in-field identification of biotic threats of wheat

“…Besides, despite there being many general and open-source software libraries and toolkits, such as OpenCV (OpenCV, 2023), TensorFlow (TensorFlow, 2023), PyTorch (PyTorch, 2023), scikit-learn (Scikit-learn, 2023), open-source and end-to-end platforms that develop computer vision systems for the agricultural domain are not so numerous. In brief, we report three examples: AirSurf, an automated and open-source analytic platform to measure yield-related phenotypes from ultra-large aerial imagery (Bauer et al, 2019); CoFly, a modular platform incorporating custom-developed AI and information and communication technologies (ICT) for unmanned aerial vehicle (UAV) applications in precision agriculture (Raptis et al, 2023); and Fiware, a general framework of open-source platform components for developing and integrating also smart farming solutions (Fiware, 2023).…”

CropQuant-Air: an AI-powered system to enable phenotypic analysis of yield- and performance-related traits using wheat canopy imagery collected by low-cost drones

“…In order to study yield performance, a range of image sensors such as red-green-blue (RGB) cameras, multi-and hyper-spectral devices, Light Detection and Ranging (LiDAR), and thermal and infrared sensors (Kachamba et al, 2016;Gracia-Romero et al, 2017;ten Harkel et al, 2020) were utilised in drone phenotyping to acquire plant's morphological and spectral features, from which yieldrelated traits and proxies could be derived (Jiang et al, 2021). For example, AirSurf applied convolutional neural networks (CNNs) to analyse millions of lettuce heads collected by manned light aircrafts, so that marketable yield of lettuce production could be estimated (Bauer et al, 2019); multi-temporal vegetation indices derived from drone-collected multi-spectral and RGB imagery were employed to predict rice grain production, showing the drone-based phenotyping could be used to identify the optimal stage for carrying out yield prediction in rice (Zhou X. et al, 2017); deep CNNs were employed to estimate rice yield performance during ripening based on aerial imagery (Yang et al, 2019); multimodal data fusion and deep learning were integrated into the classification of yield production in soybean through dronebased field phenotyping (Maimaitijiang et al, 2017); CropQuant-3D utilised open-source 3D point clouds analysis algorithms to extract canopy-level yield-related traits (e.g. 3DCI) collected by light detection and ranging (LiDAR) or drones to identify resource use efficiency wheat varieties and their yield performance (Zhu et al, 2021); AirMeasurer combined computer vision and supervised machine learning (ML) to build dynamic phenotyping algorithms to analyse yield-related traits in rice (e.g.…”