Automatic Tomato and Peduncle Location System Based on Computer Vision for Use in Robotized Harvesting

Abstract

Protected agriculture is a field in which the use of automatic systems is a key factor. In fact, the automatic harvesting of delicate fruit has not yet been perfected. This issue has received a great deal of attention over the last forty years, although no commercial harvesting robots are available at present, mainly due to the complexity and variability of the working environments. In this work we developed a computer vision system (CVS) to automate the detection and localization of fruit in a tomato crop in a typical Mediterranean greenhouse. The tasks to be performed by the system are: (1) the detection of the ripe tomatoes, (2) the location of the ripe tomatoes in the XY coordinates of the image, and (3) the location of the ripe tomatoes’ peduncles in the XY coordinates of the image. Tasks 1 and 2 were performed using a large set of digital image processing tools (enhancement, edge detection, segmentation, and the feature’s description of the tomatoes). Task 3 was carried out using basic trigonometry and numerical and geometrical descriptors. The results are very promising for beef and cluster tomatoes, with the system being able to classify 80.8% and 87.5%, respectively, of fruit with visible peduncles as “collectible”. The average processing time per image for visible ripe and harvested tomatoes was less than 30 ms.

Evaluating the Single-Shot MultiBox Detector and YOLO Deep Learning Models for the Detection of Tomatoes in a Greenhouse

“…In recent years, machine learning, and especially deep learning, techniques in fruit detection has been increasingly used and tested [8,[15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32]. Unlike conventional methods, machine learning is a more robust and accurate alternative with a better response to problems such as occlusion and green tomato detection.…”

“…These challenges become greater in the early ripening stages, due to the high colour correlation between leaves and tomatoes. Despite this fact, the most common and relevant research found in the literature considers the harvesting period in the late maturation stage of tomatoes (where the tomato is already red), so the colour is therefore a feature used recurrently to differentiate the objects to be detected [15][16][17][18][19][20]24,34]. Considering the case of fruit detection and segmentation, the authors try to distinguish it from everything external and the background, which at the crop level, can be very complex.…”

“…Considering the case of fruit detection and segmentation, the authors try to distinguish it from everything external and the background, which at the crop level, can be very complex. Several colour spaces such as Hue, Saturation and Intensity (HSI) [17,18], CIELAB (or L*a*b*) [15,16,24] and RGB [17][18][19]34] are used to extract this feature. Besides, mathematical morphology approaches [35] combined with machine learning techniques have also been used in fruit detection in occlusion and overlap situations [8,[15][16][17][18][19][20][21][22][23][24][25][26].…”

“…The results were not quantified. Benavides et al [19] designed a computer vision system for the detection of ripe tomatoes in greenhouses. The segmentation of the fruit was mainly done based on the colour and edges of the fruit, using the R component of the RGB images and the Sobel operator [37], respectively.…”

Color-Ratio Maps Enhanced Optical Filter Design and Its Application in Green Pepper Segmentation

“…Precision agriculture can provide useful information in the early stage to enable better decision making on the management system. In recent years, computer vision and artificial intelligence technology have developed to meet the growing demand for fast and accurate grain crop production [ 1 , 2 ]. As reviewed by a previous study [ 3 ], machine learning techniques have been widely used for the early and precise detection of biotic stress in the crop, specifically for the detection of weeds, plant diseases, and insect pests.…”