Computer vision-based localisation of picking points for automatic litchi harvesting applications towards natural scenarios

Zhuang, Jiajun; Hou, Chao; Tang, Yu; He, Yong; Guo, Qiwei; Zhong, Zhenyu; Luo, Shaoming

doi:10.1016/j.biosystemseng.2019.08.016

Cited by 55 publications

(26 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, it is difficult to quantitatively assess the internal mechanical damage evolution and damage volume changing of strawberries by numerical simulation. This gap also hampers the development of non-destructive methods for fruit quality analysis (Zude et al, 2006(Zude et al, , 2019 and limit the opportunities for investigating the effects of the application of external forces during harvesting (Zhuang et al, 2019), packaging and transport for recommending improved handling methods in the supply chain (Li et al, 2017b;Li and Thomas, 2014). The objectives of this study were, therefore (i) to compare the mechanical deformation behavior of strawberries at different compressibility levels; (ii) to test the effects of compression speed and loading direction on the textural failure mechanics of strawberries at different percentage deformation by loading-unloading tests; and (iii) to characterize the failure mechanics of strawberry fruit's outer and inner tissues by loading-unloading tests at different compression speeds.…”

Section: Introductionmentioning

confidence: 99%

Characterization of textural failure mechanics of strawberry fruit

Zude-Sasse

et al. 2020

Journal of Food Engineering

View full text Add to dashboard Cite

Fresh strawberry fruit is highly susceptible to damage during mechanical handlings. To prevent fruit macro-damage from external forces and predict damage evolution in internal tissues, the textural failure mechanics of strawberry fruit and its tissues were characterized by loading-unloading tests at different compression speeds. Strawberry fruit showed expected three stages of deformation during the loading phase, namely elastic, local plastic and structural failure deformation. Their cutoff points depended on the compression speed and loading direction, which was validated further by the corresponding visible browning processes in tissues from fruit longitudinal equatorial section. The peak force and absorbed energy depended on the loading direction and compression speed while the percentage of damaged mass only depended on the loading direction. The fruit was most susceptible to mechanical damage when it was compressed along its stem-blossom axis at low percentage deformation and along its radial axis at high percentage deformation. The absorbed energy and percentage of damaged mass of the strawberry fruit was correlated, which suggested that the absorbed energy could be an appropriate and easily measurable mechanical parameter for quantitatively assessing the degree of fruit damage. The failure stress, failure energy and elastic modulus of fruit tissues increased with the compression speed, while this factor did not affect the failure strain. The average failure stress, failure strain, failure energy and elastic modulus of fruit inner tissue were 0.093

show abstract

Section: Introductionmentioning

confidence: 99%

Characterization of textural failure mechanics of strawberry fruit

Zude-Sasse

et al. 2020

Journal of Food Engineering

View full text Add to dashboard Cite

show abstract

“…CogPMAlignTool was employed to identify four marginal fruits of a mature bunch to guide the robot to finish the task indirectly, yielding an 83% successful harvest. In another study [29], an acceptable method was proposed to locate the picking point of a litchi cluster via application of different color spaces when one image contained only a bunch of fruits, with an 83% accuracy rate and near-real-time results being yielded. Some of the abovementioned studies needed either a cue or ideal conditions for good performance to be achieved.…”

Section: Introductionmentioning

confidence: 99%

Detection of Fruit-Bearing Branches and Localization of Litchi Clusters for Vision-Based Harvesting Robots

et al. 2020

View full text Add to dashboard Cite

Litchi clusters in fruit groves are randomly scattered and occur irregularly, so it is difficult to detect and locate the fruit-bearing branches of multiple litchi clusters at one time. This is a highly challenging task related to continuous operation in the natural environment for visual-based harvesting robots to carry out. In this study, a reliable algorithm based on RGB-depth (RGB-D) cameras in the fields was developed to accurately and automatically detect and locate the fruit-bearing branches of multiple litchi clusters simultaneously in large environments. A semantics segmentation method, Deeplabv3, was employed to segment the RGB images into three categories: background, fruit and twig. A pre-processing step is proposed to align the segmented RGB images and remove the twigs that did not bear fruits. Subsequently, the twig binary map image was processed via skeleton extraction and pruning operations, which left behind only the main branches of twigs. A method for non-parametric density-based spatial clustering of application with noise was used to cluster the pixels in the three-dimensional space of the skeleton map of the branches; thus, the fruit-bearing branches belonging to the same litchi clusters were determined. Finally, a three-dimensional straight line was fitted to each cluster via principal component analysis, and the linear information corresponded to the location of the fruit-bearing branches. In the experiments, 452 pairs of RGB-D images under different illumination were collected to test the proposed algorithm. The results show that the detection accuracy of a litchi fruit-bearing branch is 83.33%, positioning accuracy is 17.29º ±24.57º , and execution time for the determination of a single litchi fruit-bearing branch is 0.464s. Field experiments show that this method can effectively guide the robot to complete continuous picking tasks. INDEX TERMS Continuous picking, location detection of fruit-bearing branches, harvesting robots, RGB-D image

show abstract

“…Spatial information needs to be known with respect to the local or global reference system to localize the agricultural vehicle and operating targets. For relative positioning in a local area, a variety of sensors were used to including laser range finder, computer vision and 3D camera [6][7][8][9][10] . For localization in the field, absolute positioning is preferred especially when there exist no reference objects.…”

Section: Introduction mentioning

confidence: 99%

Development of autonomous navigation controller for agricultural vehicles

Yin

Wang

Chen

et al. 2020

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

Agricultural vehicles are adopted to undertake farming tasks by traversing along crop rows in the field. Working quality depends significantly on the driving skills of the operator. Automatic guidance has been introduced into agriculture to achieve high-accuracy path tracking during the last decades, which contributes considerably to straight-line navigation. The objective of this research was to develop an autonomous navigation controller that allowed movement autonomy for various agricultural vehicles. Three wheel-type vehicles were used as the test platform featuring automatic steering, hydrostatic transmission and speed control, which included a rice transplanter, a high-clearance sprayer and a tractor. A dual-antenna RTK-GNSS receiver was attached to the vehicles to provide spatial information on both positioning and heading by using the RTX service from Trimble. A path planning method was proposed to create a straight-line reference path by giving two points, and the target path was determined according to the vehicle initial status and working assignment. Headland turning was comprehensively taken into account by listing different turn patterns in order to realize autonomous navigation at the headland. The navigation controller hardware was fabricated for program execution, data processing and information communication with peripherals. A human-machine interface was designed for the operator to complete basic setting, path planning and navigation control by providing controls. Field experiments were conducted to evaluate the performance and versatility of the newly developed autonomous navigation controller in guiding agricultural vehicles to follow straight paths and turn at the headland. Results showed that an appropriate turn pattern was automatically executed when finishing straight-line navigation. The lateral error in straight-line tracking was no more than 6 cm, 6 cm and 5 cm for the rice transplanter, the high-clearance sprayer and the tractor, respectively. And the maximum lateral RMS error was 3.10 cm, 4.75 cm, 2.21 cm in terms of straight-line tracking, which indicated that the newly developed autonomous navigation controller was versatile and of high robustness in guiding various agricultural vehicles.

show abstract

Computer vision-based localisation of picking points for automatic litchi harvesting applications towards natural scenarios

Cited by 55 publications

References 20 publications

Characterization of textural failure mechanics of strawberry fruit

Characterization of textural failure mechanics of strawberry fruit

Detection of Fruit-Bearing Branches and Localization of Litchi Clusters for Vision-Based Harvesting Robots

Development of autonomous navigation controller for agricultural vehicles

Contact Info

Product

Resources

About