Fruit Mapping with Shape Completion for Autonomous Crop Monitoring

Abstract

Although inverse kinematics of serial manipulators is a well studied problem, challenges still exist in finding smooth feasible solutions that are also collision aware. Furthermore, with collaborative and service robots gaining traction, different robotic systems have to work in close proximity. This means that the current inverse kinematics approaches have to not only avoid collisions with themselves but also collisions with other robot arms. Therefore, we present a novel approach to compute inverse kinematics for serial manipulators that take into account different constraints while trying to reach a desired end-effector position and/or orientation that avoids collisions with themselves and other arms. Unlike other constraint based approaches, we neither perform expensive inverse Jacobian computations nor do we require arms with redundant degrees of freedom. Instead, we formulate different constraints as weighted cost functions to be optimized by a non-linear optimization solver. Our approach is superior to the state-of-the-art CollisionIK in terms of collision avoidance in the presence of multiple arms in confined spaces with no detected collisions at all in all the experimental scenarios. When the probability of collision is low, our approach shows better performance at trajectory tracking as well. Additionally, our approach is capable of simultaneous yet decentralized control of multiple arms for trajectory tracking in intersecting workspace without any collisions.

Immature Green Apple Detection and Sizing in Commercial Orchards Using YOLOv8 and Shape Fitting Techniques

“…Despite their innovative approaches, these methods face challenges with computational demands and occlusions, leading to incomplete reconstructions. Subsequent solutions by [52]and [53] investigated automated shape completion, fitting ellipsoids to gathered point clouds. Yet, they faced challenges due to the plant's dynamic structure or the computational load.…”

Leveraging I4.0 smart methodologies for developing solutions for harvesting produce

“…They created a Gazebo simulation environment of the cotton field to test the identification capabilities of multiple LiDAR configurations from 3D point cloud data, which was converted into voxel grids, for navigation in the crop rows. Another study used Gazebo to model a sweet pepper environment with a UR5e robot arm to evaluate the accuracy of their fruit shape estimation approach based on super ellipsoids (Marangoz et al, 2022). Their simulation was integrated with an octree-based truncated signed distance field to map the images collected by the robot's RGB camera.…”

“…These model types have been applied in agriculture. For example, an octree-based map in the form of TSDF for a sweet pepper environment was developed by Marangoz et al (2022) to estimate produce shapes.…”

Combining Local and Global Viewpoint Planning for Fruit Coverage

“…We aim to show the fruit detection performance of our VMP approach compared to the state-of-the-art RVP method after a fixed mission time. To evaluate the fruit detection performance of the two planners, we used our previously developed sweet pepper shape estimator (Marangoz et al [26]). The shape estimator runs Voxblox [17] on the extracted fruit point clouds to obtain an accurate fruit map, then clusters the obtained surface point cloud and matches superellipsoids to estimate the fruit shapes and positions.…”

“…While no fruit ground truth is available, we qualitatively validate that our approach plans view poses monitoring the majority of fruit clusters. Utilizing our clustering and fruit shape estimation approach [26], the resulting fruit map could be leveraged in downstream tasks, e.g. yield estimation.…”