3D Perception-Based Collision-Free Robotic Leaf Probing for Automated Indoor Plant Phenotyping

Abstract

Various instrumentation devices for plant physiology study such as spectrometer, chlorophyll fluorimeter, and Raman spectroscopy sensor require accurate placement of their sensor probes toward the leaf surface to meet specific requirements of probe-to-target distance and orientation. In this work, a Kinect V2 sensor, a highprecision 2D laser profilometer, and a six-axis robotic manipulator were used to automate the leaf probing task. The relatively wide field of view and high resolution of Kinect V2 allowed rapid capture of the full 3D environment in front of the robot. The location and size of each plant were estimated by k-means clustering where "k" was the user-defined number of plants. A real-time collision-free motion planning framework based on Probabilistic Roadmaps was adapted to maneuver the robotic manipulator without colliding with the plants. Each plant was scanned from the top with the short-range profilometer to obtain high-precision 3D point cloud data. Potential leaf clusters were extracted by a 3D region growing segmentation scheme. Each leaf segment was further partitioned into small patches by a Voxel Cloud Connectivity Segmentation method. Only the patches with low root mean square errors of plane fitting were used to compute leaf probing poses of the robot. Experiments conducted inside a growth chamber mock-up showed that the developed robotic leaf probing system achieved an average motion planning time of 0.4 seconds with an average end-effector travel distance of 1.0 meter. To examine the probing accuracy, a square surface was scanned at different angles, and its centroid was probed perpendicularly. The average absolute probing errors of distance and angle were 1.5 mm and 0.84 degrees, respectively. These results demonstrate the utility of the proposed robotic leaf probing system for automated non-contact deployment of spectroscopic sensor probes for indoor plant phenotyping under controlled environmental conditions. This manuscript is in press. It has been accepted for publication in Transactions of the ASABE. When the final, edited version is posted online this in-press version will be removed. Example citation: Authors. Year. Article title. Trans. ASABE (in press). DOI number. The DOI for this manuscript is https://doi.org/10.13031/trans.12653. It will remain the same after publication. ABSTRACT. Various instrumentation devices for plant physiology study such as spectrometer, chlorophyll fluorimeter, and Raman

Assessing plant performance in the Enviratron

“…As such, multiple probing locations may be found on a single maize leaf. To accommodate different potential species of interest, the parameters of the 3D region growing segmentation can be adjusted so that leaf segments of various sizes can be produced [7].…”

“…Leaf segmentation is performed on this point cloud. Finally, fluorometer data and hyperspectral line scans are acquired on leaf surfaces if a collision-free path exists [7,8].…”

Augmented Perception for Agricultural Robots Navigation

“…In comparison, stereo vision provides -at present-more resolution (number of pixels or points), but TOF sensors are active sensor with their own illumination source, and therefore they are independent of natural illumination, working both during the day and at night. In an experiment conducted in a laboratory setting, a 3D Kinect sensor was used to operate a robotic manipulator to sample leaves, using a 3D occupancy grid to find collision-free paths [9]. Even though 3D perception provides a faithful reconstruction of the surrounding environment as a result of a point cloud, where each point can be well determined by its three Cartesian coordinates (x, y, z), monocular vision is also capable of retrieving the visual cues needed to assist an autonomous vehicle.…”

“…After the six regions of the grid have been mathematically defined by (8), the subsequent geometrical parameters are calculated for each specific region omij. The first such parameter is the cumulative profile CUMij(h) defined in (9):…”

“…(9) The cumulative profile of (9) was used to calculate the moment Mij, whose expression for the first zone M11 is given in (10), being the rest of moments Mij for the rest of the zones easily deducible using the same procedure as in (9) and (10). Associated with the moment, and following the same philosophy, the summation of the cumulative profile SUM11 was calculated according to (11).…”