Window Zooming–Based Localization Algorithm of Fruit and Vegetable for Harvesting Robot

Chen, Zhong; Luo, Tianhong; Zhao, Lijun; Tang, Yunchao; Zou, Xiangjun

doi:10.1109/access.2019.2925812

Cited by 34 publications

(20 citation statements)

References 27 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More recently, Wang et al [6] proposed an improvement in the methods of fruit localization for harvesting robots using binocular stereo images. The proposed method uses the Faster region-based convolutional neural network (R-CNN), which is able to achieve a recognition rate of 96.33% in six different conditions.…”

Section: Methodsmentioning

confidence: 99%

“…These can include spaceborne, airborne and terrestrial platforms, with applications in crop monitoring, water management, fertigation, harvesting, and decision making in general. In particular, using automated or semi-automated harvesting methods is not only economically cost effective, but also more efficient than traditional methods, and significantly reduces the harvest More recently, Wang et al [6] proposed an improvement in the methods of fruit localization for harvesting robots using binocular stereo images. The proposed method uses the Faster region-based convolutional neural network (R-CNN), which is able to achieve a recognition rate of 96.33% in six different conditions.…”

Section: Introductionmentioning

confidence: 99%

“…This is a breakthrough compared to other previous works of the authors, such as [17] for the detection of grape clusters in vineyards, [18] for green grape, and [19,20] for the case of litchi fruits. An interesting description of other current research works in fruit detection and localization can be found in [6].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Comparison of Different Classifiers and the Majority Voting Rule for the Detection of Plum Fruits in Garden Conditions

et al. 2019

View full text Add to dashboard Cite

Color segmentation is one of the most thoroughly studied problems in agricultural applications of remote image capture systems, since it is the key step in several different tasks, such as crop harvesting, site specific spraying, and targeted disease control under natural light. This paper studies and compares five methods to segment plum fruit images under ambient conditions at 12 different light intensities, and an ensemble method combining them. In these methods, several color features in different color spaces are first extracted for each pixel, and then the most effective features are selected using a hybrid approach of artificial neural networks and the cultural algorithm (ANN-CA). The features selected among the 38 defined channels were the b* channel of L*a*b*, and the color purity index, C*, from L*C*h. Next, fruit/background segmentation is performed using five classifiers: artificial neural network-imperialist competitive algorithm (ANN-ICA); hybrid artificial neural network-harmony search (ANN-HS); support vector machines (SVM); k nearest neighbors (kNN); and linear discriminant analysis (LDA). In the ensemble method, the final class for each pixel is determined using the majority voting method. The experiments showed that the correct classification rate for the majority voting method excluding LDA was 98.59%, outperforming the results of the constituent methods.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparison of Different Classifiers and the Majority Voting Rule for the Detection of Plum Fruits in Garden Conditions

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Koirala et al [38] proposed the MangoYOLO network for real-time detection of mangoes in orchards with high accuracy and in real time. Wang et al [39] used the Faster Region-based Convolutional Neural Network (R-CNN) model to identify fruits and vegetables. Gao et al [40] proposed an apple detection method based on a fast regional convolutional neural network for multiclass apple dense fruit trees.…”

Section: Introductionmentioning

confidence: 99%

YOLOv3-Litchi Detection Method of Densely Distributed Litchi in Large Vision Scenes

Wang

Dong

Zhou

et al. 2021

Mathematical Problems in Engineering

Self Cite

View full text Add to dashboard Cite

Accurate and reliable fruit detection in the orchard environment is an important step for yield estimation and robotic harvesting. However, the existing detection methods often target large and relatively sparse fruits, but they cannot provide a good solution for small and densely distributed fruits. This paper proposes a YOLOv3-Litchi model based on YOLOv3 to detect densely distributed litchi fruits in large visual scenes. We adjusted the prediction scale and reduced the network layer to improve the detection ability of small and dense litchi fruits and ensure the detection speed. From flowering to 50 days after maturity, we collected a total of 266 images, including 16,000 fruits, and then used them to construct the litchi dataset. Then, the k-means++ algorithm is used to cluster the bounding boxes in the labeled data to determine the priori box size suitable for litchi detection. We trained an improved YOLOv3-Litchi model, tested its litchi detection performance, and compared YOLOv3-Litchi with YOLOv2, YOLOv3, and Faster R-CNN on the actual detection effect of litchi and used the F1 value and the average detection time as the assessed value. The test results show that the F1 of YOLOv3-Litchi is higher than that of YOLOv2 algorithm 0.1, higher than that of YOLOv3 algorithm 0.08, and higher than that of Faster R-CNN algorithm 0.05; the average detection time of YOLOv3-Litchi is 29.44 ms faster than that of YOLOv2 algorithm, 19.56 ms faster than that of YOLOv3 algorithm ms, and 607.06 ms faster than that of Faster R-CNN algorithm. And the detection speed of the improved model is faster. The proposed model remits optimal detection performance for small and dense fruits. The work presented here may provide a reference for further study on fruit-detection methods in natural environments.

show abstract

“…A 93.7% success rate of detection was yielded for fruit calyxes, and a 92% success rate was yielded for the correct segregation of fruits during nighttime with flash. In [17], two CCD color cameras integrated with a window zooming-based algorithm were utilized to locate multiple fruits and vegetables. The results showed that under varying illumination and partially occluded circumstances, the localization errors were less than 7.5 mm for a measuring distance of 300-1600mm.…”

Section: Introductionmentioning

confidence: 99%

Detection of Fruit-Bearing Branches and Localization of Litchi Clusters for Vision-Based Harvesting Robots

et al. 2020

Self Cite

View full text Add to dashboard Cite

Litchi clusters in fruit groves are randomly scattered and occur irregularly, so it is difficult to detect and locate the fruit-bearing branches of multiple litchi clusters at one time. This is a highly challenging task related to continuous operation in the natural environment for visual-based harvesting robots to carry out. In this study, a reliable algorithm based on RGB-depth (RGB-D) cameras in the fields was developed to accurately and automatically detect and locate the fruit-bearing branches of multiple litchi clusters simultaneously in large environments. A semantics segmentation method, Deeplabv3, was employed to segment the RGB images into three categories: background, fruit and twig. A pre-processing step is proposed to align the segmented RGB images and remove the twigs that did not bear fruits. Subsequently, the twig binary map image was processed via skeleton extraction and pruning operations, which left behind only the main branches of twigs. A method for non-parametric density-based spatial clustering of application with noise was used to cluster the pixels in the three-dimensional space of the skeleton map of the branches; thus, the fruit-bearing branches belonging to the same litchi clusters were determined. Finally, a three-dimensional straight line was fitted to each cluster via principal component analysis, and the linear information corresponded to the location of the fruit-bearing branches. In the experiments, 452 pairs of RGB-D images under different illumination were collected to test the proposed algorithm. The results show that the detection accuracy of a litchi fruit-bearing branch is 83.33%, positioning accuracy is 17.29º ±24.57º , and execution time for the determination of a single litchi fruit-bearing branch is 0.464s. Field experiments show that this method can effectively guide the robot to complete continuous picking tasks. INDEX TERMS Continuous picking, location detection of fruit-bearing branches, harvesting robots, RGB-D image

show abstract

Window Zooming–Based Localization Algorithm of Fruit and Vegetable for Harvesting Robot

Cited by 34 publications

References 27 publications

Comparison of Different Classifiers and the Majority Voting Rule for the Detection of Plum Fruits in Garden Conditions

Comparison of Different Classifiers and the Majority Voting Rule for the Detection of Plum Fruits in Garden Conditions

YOLOv3-Litchi Detection Method of Densely Distributed Litchi in Large Vision Scenes

Detection of Fruit-Bearing Branches and Localization of Litchi Clusters for Vision-Based Harvesting Robots

Contact Info

Product

Resources

About