Evaluating the Single-Shot MultiBox Detector and YOLO Deep Learning Models for the Detection of Tomatoes in a Greenhouse

Abstract

The development of robotic solutions for agriculture requires advanced perception capabilities that can work reliably in any crop stage. For example, to automatise the tomato harvesting process in greenhouses, the visual perception system needs to detect the tomato in any life cycle stage (flower to the ripe tomato). The state-of-the-art for visual tomato detection focuses mainly on ripe tomato, which has a distinctive colour from the background. This paper contributes with an annotated visual dataset of green and reddish tomatoes. This kind of dataset is uncommon and not available for research purposes. This will enable further developments in edge artificial intelligence for in situ and in real-time visual tomato detection required for the development of harvesting robots. Considering this dataset, five deep learning models were selected, trained and benchmarked to detect green and reddish tomatoes grown in greenhouses. Considering our robotic platform specifications, only the Single-Shot MultiBox Detector (SSD) and YOLO architectures were considered. The results proved that the system can detect green and reddish tomatoes, even those occluded by leaves. SSD MobileNet v2 had the best performance when compared against SSD Inception v2, SSD ResNet 50, SSD ResNet 101 and YOLOv4 Tiny, reaching an F1-score of 66.15, an mAP of 51.46 and an inference time of 16.44ms with the NVIDIA Turing Architecture platform, an NVIDIA Tesla T4, with 12 GB. YOLOv4 Tiny also had impressive results, mainly concerning inferring times of about 5ms.

Neural Network-Based Price Tag Data Analysis

“…Comparison of YOLOv4-tiny with SSD and the faster R-CNN showed the following results: SSD had an F1 score 0.67 and the faster R-CNN had an F1 score 0.66 according to article of AN Veeranampalayam Sivakumar [30], while our realization of YOLOv4-Tiny had an F1 score of 0.61. However, various studies show that YOLOv4-tiny can outperform SSD [31,32] and the faster R-CNN [24,33]. Additionally, YOLOv4-tiny has a higher FPS, which leads to faster performance [24,33].…”

Unimodal and Multimodal Perception for Forest Management: Review and Dataset

“…The majority of them are about autonomous harvesting where the fruit or vegetable must be detected and/or segmented prior to its picking [101][102][103][104][105][106][107][108][109]. The detection of vegetables or fruits are also important to count them and estimate the production yield [110][111][112][113]. Similarly to forestry contexts, some works are about disease detection and monitoring [114,115], and others are focused on detection woody trunks, weeds, and general obstacles in crops for navigation [116][117][118][119], operation purposes [120,121], and cleaning tasks [122,123].…”

The Intelligent Path Planning System of Agricultural Robot via Reinforcement Learning

“…In the study by [7], they review the IoT systems in agriculture, and analyze the status and challenges of automatic harvesting robots and automatic picking robots. Additionally, ref [8] uses CNN and SSD model to detect the life cycle of tomatoes in greenhouses and improves the perception ability of picking robots. The above research reflects the necessity and effectiveness of artificial intelligence technology in the agricultural field, and most task-oriented agricultural robots are inseparable from the problem of autonomous path planning.…”