Localization of fruit and vegetable is of great significance to fruit and vegetable harvesting robots and even harvesting industries. However, uncontrollable factors, such as varying illumination, random occlusion, and various surface color and texture, constrain the localization of fruit and vegetable using the vision imaging technology under unconstructed environment. Our previous studies have developed various methods (illumination normalization, features-based classification, etc.) to localize a certain kind of fruit or vegetable using the binocular stereo vision. However, the localization of the multiple fruit and vegetable still faces challenges due to the uncontrollable factors. In order to address this issue, this study proposed an intelligent localization method of targets in fruit and vegetable images acquired by the two charge-coupled device (CCD) color cameras under unstructured environment. The method utilized the Faster region-based convolutional neural network (R-CNN) model to recognize the fruit and vegetable. Based on the recognition results, a window zooming method was proposed for the matching of the recognized target. Finally, the localization of the target was completed by calculating the three-dimensional coordinates of the matched target using the triangular measurement principle. The experimental results showed that the proposed method could be robust against the influences of varying illumination and occlusion, and the average accurate recognition rate was 96.33% under six different conditions. About 93.44% of 1036 pairs of tested targets from unoccluded and partially occluded conditions were successfully matched. Localization errors had no significant difference and they were less than 7.5 mm when the measuring distance was between 300 and 1600 mm under varying illumination and partially occluded conditions.INDEX TERMS Fruit and vegetable localization, vision imaging technology, binocular stereo vision, unstructured environment.
With the development of the fruit farming industry, there have been breakthroughs in both scale and harvesting requirements, and the resulting problem is that the demand of picking robot is more and more high, the function, cost and quality of the harvest and picking efficiency compared with the traditional manual operation mode with strong competitiveness for market, that is to say, in terms of cost cheaper, picking at a faster rate, and can avoid damage on the fruit in the process of picking. Therefore, in terms of mechanical structure design, the transmission accuracy and efficiency of the picking robot should be improved. At the same time, the structural design of the picking robot should simplify the structure as much as possible, reduce the manufacturing cost, and ensure the feasibility of the picking robot’s functions. Picking robot combined with automatic walking system, automatic detection system and control system-centered intelligent system, can achieve accurate and efficient work in various conditions. Realize picking robot toward automation, intelligence, scientific development. The design of this topic is mainly aimed at the development of the picking robot and the use of analysis, determine the feasibility of the picking robot design and virtual simulation.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.