Aiming at the problem of apple branch obstacle localization in fruit picking process of harvesting robot manipulator, in order to obtain three-dimensional information of the apple branch obstacle, the binocular stereo vision localization method of apple branch obstacle is proposed. Firstly, branch skeleton is extracted by morphological thinning method and then the feature skeleton is obtained after removing the false branch and recovering the occluded branch. After that, the endpoints and bifurcation points regarded as match feature points are extracted from skeleton, and the stereo matching algorithm based on features is adopted. Then, the depth information of branch obstacle is obtained on the basis of triangulation theory. Finally, the experiment results for apple tree branches localization show that the error lies in ±6.2 mm. Moreover, the error is merely ±1.5 mm when the distance between the object and the binocular camera is 1000 mm, which meets with localization accuracy requirements of apple harvesting robot visual system.
According to the grasping damage of apple during the process of robot picking apple, the variation of interior tensions inside the apple skin in the grasping process of apple with different type finger of robot end‐effector is researched. The finite element model for apple is established by ANSYS. Some simulations for the grasping process of apple with plane and arc‐shaped finger are carried out. The Von Mises stress nephograms of apple different tissue under different load force by different type fingers are obtained. The experimental results show that the apple cortex is more easily to get damaged due to its small failure stress. And the deformation and stress of apple caused by arc‐shaped finger are smaller than by plane finger. At last, the actual experiment for apple grasping damage of end‐effector with arc‐shaped finger validates the reliability of simulated results. The research results demonstrated that the finite element method can make accurate evaluation for apple damage.
Practical applications
A major problem associated with robot harvesting is the mechanical damage of apple caused by end‐effector of robot. When the apple is grasped by the end‐effector, the mechanical damage is often occurred underneath apple skin thus, which is difficult to find by the naked eye immediately. The results in our paper can accurate evaluation for apple damage and provide a foundational basis to develop an injury‐reduce device of apple harvesting robot.
In order to improve the working efficiency of robot promptly picking ripe apples, the harvesting robot must have the ability of continuous recognition and operation at night. Nighttime apple image has many dark spaces and shadows with low resolution, and therefore, a Retinex algorithm based on guided filter is presented to enhance nighttime image in this article. According to color feature of image, the illumination component is estimated by using guided filter which can be applied as an edge-preserving smoothing operator. And the reflection component with image itself characteristics is obtained by employing single-scale Retinex algorithm. After gamma correction, these two components of image are synthesized into a new enhanced nighttime apple image. Fifty nighttime images acquired under fluorescent lighting are selected to make experiment. Experimental results show that the image enhancement performance indexes processed by the proposed algorithm, such as average gray value, standard deviation, information entropy, average gradient, and segmentation error are superior to those of histogram equalization algorithms and Retinex algorithm based on bilateral filter. In addition, compared with the Retinex algorithm based on bilateral filter, the proposed algorithm has an average reduction of 74.56% in running time with better real-time and higher efficiency. So it can realize the continuous operation of apple harvesting robot at night.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.