Single-Shot Convolution Neural Networks for Real-Time Fruit Detection Within the Tree

Abstract

Image/video processing for fruit detection in the tree using hard-coded feature extraction algorithms has shown high accuracy on fruit detection during recent years. While accurate, these approaches even with high-end hardware are still computationally intensive and too slow for real-time systems. This paper details the use of deep convolution neural networks architecture based on single-stage detectors. Using deep-learning techniques eliminates the need for hard-code specific features for specific fruit shapes, color and/or other attributes. This architecture takes the input image and divides into AxA grid, where A is a configurable hyper-parameter that defines the fineness of the grid. To each grid cell an image detection and localization algorithm is applied. Each of those cells is responsible to predict bounding boxes and confidence score for fruit (apple and pear in the case of this study) detected in that cell. We want this confidence score to be high if a fruit exists in a cell, otherwise to be zero, if no fruit is in the cell. More than 100 images of apple and pear trees were taken. Each tree image with approximately 50 fruits, that at the end resulted on more than 5000 images of apple and pear fruits each. Labeling images for training consisted on manually specifying the bounding boxes for fruits, where (x, y) are the center coordinates of the box and (w, h) are width and height. This architecture showed an accuracy of more than 90% fruit detection. Based on correlation between number of visible fruits, detected fruits on one frame and the real number of fruits on one tree, a model was created to accommodate this error rate. Processing speed is higher than 20 FPS which is fast enough for any grasping/harvesting robotic arm or other real-time applications. HIGHLIGHTS Using new convolutional deep learning techniques based on single-shot detectors to detect and count fruits (apple and pear) within the tree canopy.

Applications of Deep Learning for Dense Scenes Analysis in Agriculture: A Review

“…The other is the case that the background may be dynamic, which changes with the moving of the object. Images obtained from realistic agricultural scenes such as farmland or orchard [31] is licensed under https://creativecommons.org/licenses/by/4.0/; (b) Green Spike and Green Canopy (GSGC), Green Spike and Yellow Canopy (GSYC), Yellow Spike and Yellow Canopy (YSYC). Source: [32] is licensed under https://creativecommons.org/licenses/by/4.0/.…”

“…Even in datasets that are significantly different from training data, the optimal recall rate and accuracy rate are close to 80%. Different from the above two-stage model, Bresilla et al [31] proposed an end-to-end fast and accurate fruit detection model based on YOLO. By modifying the standard model, the network scale was expanded, some layers were deleted, and then two other blocks are added.…”

Identification of grapevine(Vitis vinifera L.)cultivars by vine leaf image via deep learning and mobile devices

“…Therefore, more advanced approaches are desirable for an automatic genotype classification. network (CNN) [11][12][13][14][15][16], have been drawing an increasing interest in both academia and industry due to their promising performance over conventional machine learning algorithms. In particular, the improved performance is mainly due to the complex structure of CNN, the substantially increased volume of dataset and the significantly improved computation power.…”

Learning from Synthetic Dataset for Crop Seed Instance Segmentation

“…Such deep-learning-based image analysis has also been influencing the field of agriculture. This involves image-based phenotyping including weed detection 9 , crop disease diagnosis 10,11 , fruit detection 12 , and many other applications as listed in the recent review 13 . Meanwhile, not only features from images but also with that of environmental variables, functionalized a neural network to predict plant water stress for automated control of greenhouse tomato irrigation 14 .…”