Robotic Complex for Harvesting Apple Crops

Krakhmalev, Oleg; Gataullin, Sergey; Boltachev, Eldar; Korchagin, Sergey; Blagoveshchensky, Ivan; Kang, Liang

doi:10.3390/robotics11040077

Cited by 5 publications

(1 citation statement)

References 41 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of the integrated sensing system was evaluated under static conditions. This system can be combined with different robotic systems, single robotic arms [ 47 ], and parallel robotic arm systems [ 48 ] to improve the harvesting accuracy; thus, this approach can be extended to dynamic localization for faster robotic applications. One of the limitations of this system was the obstructions from the leaves, branches, and metal poles, which generated false depth values.…”

Section: Discussionmentioning

confidence: 99%

3D Camera and Single-Point Laser Sensor Integration for Apple Localization in Spindle-Type Orchard Systems

Abeyrathna,

Nakaguchi,

Liu

et al. 2024

Sensors

View full text Add to dashboard Cite

Accurate localization of apples is the key factor that determines a successful harvesting cycle in the automation of apple harvesting for unmanned operations. In this regard, accurate depth sensing or positional information of apples is required for harvesting apples based on robotic systems, which is challenging in outdoor environments because of uneven light variations when using 3D cameras for the localization of apples. Therefore, this research attempted to overcome the effect of light variations for the 3D cameras during outdoor apple harvesting operations. Thus, integrated single-point laser sensors for the localization of apples using a state-of-the-art model, the EfficientDet object detection algorithm with an mAP@0.5 of 0.775 were used in this study. In the experiments, a RealSense D455f RGB-D camera was integrated with a single-point laser ranging sensor utilized to obtain precise apple localization coordinates for implementation in a harvesting robot. The single-point laser range sensor was attached to two servo motors capable of moving the center position of the detected apples based on the detection ID generated by the DeepSORT (online real-time tracking) algorithm. The experiments were conducted under indoor and outdoor conditions in a spindle-type apple orchard artificial architecture by mounting the combined sensor system behind a four-wheel tractor. The localization coordinates were compared between the RGB-D camera depth values and the combined sensor system under different light conditions. The results show that the root-mean-square error (RMSE) values of the RGB-D camera depth and integrated sensor mechanism varied from 3.91 to 8.36 cm and from 1.62 to 2.13 cm under 476~600 lx to 1023~1100 × 100 lx light conditions, respectively. The integrated sensor system can be used for an apple harvesting robotic manipulator with a positional accuracy of ±2 cm, except for some apples that were occluded due to leaves and branches. Further research will be carried out using changes in the position of the integrated system for recognition of the affected apples for harvesting operations.

show abstract

Section: Discussionmentioning

confidence: 99%