Weed Instance Segmentation from UAV ortho-mosaic Images based on Deep Learning

Abstract

Weeds significantly impact agricultural production, and traditional weed control methods often harm soil health and the environment. This study aims to develop deep learning based segmentation models in identifying weeds in potato fields captured by Unmanned Aerial Vehicle (UAV) orthophotos and to explore the effects of weeds on potato yield. UAVs were used to collect RGB data from potato fields, flying at an altitude of 10m, with Real ESRGAN Super-Resolution (SR) enhancing image resolution. We applied the Segment Anything Model (SAM) to do semi-automatic annotation, followed by training the YOLOv8 and MASK-RCNN models for segmentation. Also we used ANOVA and linear regression to analyze the effects of weeds and nitrogen fertilizer on yield. Results showed that the detection accuracy mAP50 scores for YOLOv8 and Mask R-CNN were 0.902 and 0.920, respectively, with the Real-ESRGAN-enhanced model achieving 0.909. When multiple weed types were present, accuracy decreased to 0.86. The linear regression model, incorporating plant and weed coverage areas, explained 41.2% of yield variation (R2 = 0.412, p-value = 0.01). Both YOLOv8 and Mask R-CNN achieved high accuracy, with YOLOv8 converging faster. Real-ESRGAN reconstruction slightly improved accuracy. Different nitrogen fertilizer treatments had no significant effect on yield, while weed control treatments significantly impacted yield, showing the importance of precise weed mapping. This study provides insights into weed segmentation and contributes to environmentally friendly precision weed control.