Machine Learning Approaches to Improve Three Basic Plant Phenotyping Tasks Using Three-Dimensional Point Clouds

Abstract

Developing automated methods to efficiently process large volumes of point cloud data remains a challenge for threedimensional (3D) plant phenotyping applications. Here, we describe the development of machine learning methods to tackle three primary challenges in plant phenotyping: lamina/stem classification, lamina counting, and stem skeletonization. For classification, we assessed and validated the accuracy of our methods on a dataset of 54 3D shoot architectures, representing multiple growth conditions and developmental time points for two Solanaceous species, tomato (Solanum lycopersicum cv 75 m82D) and Nicotiana benthamiana. Using deep learning, we classified lamina versus stems with 97.8% accuracy. Critically, we also demonstrated the robustness of our method to growth conditions and species that have not been trained on, which is important in practical applications but is often untested. For lamina counting, we developed an enhanced region-growing algorithm to reduce oversegmentation; this method achieved 86.6% accuracy, outperforming prior methods developed for this problem. Finally, for stem skeletonization, we developed an enhanced tip detection technique, which ran an order of magnitude faster and generated more precise skeleton architectures than prior methods. Overall, our improvements enable higher throughput and accurate extraction of phenotypic properties from 3D point cloud data. Emerging technologies for high-throughput plant architecture capture has increased the demand for automated phenotyping methods (Furbank and Tester, 2011; Fiorani and Schurr, 2013). Extracting plant features is a first step toward quantifying biomass and yield (

Skeletonization of Plant Point Cloud Data in Stochastic Optimization Framework

“…As shown in the middle of Figure 1, sometimes skeleton points do not lie within the enclosing boundary of the original point cloud data [27], [7]. This results in invalid geometry estimation of the input data.…”

“…Left: The problem of "zigzag" structure, where the skeleton does not follow the centerline of the stem and tends to deviate towards the branching point [15] (only the main stem skeleton is shown in the figure). Middle: The problem of biologically irrelevant skeleton points which falls beyond the boundary of the input data (shown at the top part), and inability to capture the geometric details for some branches [27], [7]. Right: Overlooking tiny geometrical structures [15].…”

“…In recent years, 3D point cloud based analysis is getting extremely popular in phenotyping and agricultural applications [6]. Typical applications of point cloud based phenotyping include plant organ segmentation [7], robotic branch pruning [8], automated growth analysis [9], etc. Many of these applications require skeleton structure of the input point cloud data as a prior for further processing.…”

Skeletonization of Plant Point Cloud Data Using Stochastic Optimization Framework

“…In recent years, 3D point cloud based analysis is getting extremely popular in phenotyping and agricultural applications (Vázquez-Arellano et al, 2016). Typical applications of point cloud based phenotyping include plant organ segmentation (Ziamtsov and Navlakha, 2019), robotic branch pruning (Chattopadhyay et al, 2016), automated growth analysis (Chaudhury et al, 2019), etc. Many of these applications require skeleton structure of the input point cloud data as a prior for further processing.…”

“…(Left) The problem of zigzag structure, where the skeleton does not follow the centerline of the stem and tends to deviate toward the branching point (Xu et al, 2007) (only the main stem skeleton is shown in the figure). (Middle) The problem of biologically irrelevant skeleton points which falls beyond the boundary of the input data (shown at the top part), and inability to capture the geometric details for some branches (Delagrange et al, 2014;Ziamtsov and Navlakha, 2019). (Right) Overlooking tiny geometrical structures (Xu et al, 2007).…”