A Low-Cost 3D Phenotype Measurement Method of Leafy Vegetables Using Video Recordings from Smartphones

Abstract

Leafy vegetables are an essential source of the various nutrients that people need in their daily lives. The quantification of vegetable phenotypes and yield estimation are prerequisites for the selection of genetic varieties and for the improvement of planting methods. The traditional method is manual measurement, which is time-consuming and cumbersome. Therefore, there is a need for efficient and convenient in situ vegetable phenotype identification methods to provide data support for breeding research and for crop yield monitoring, thereby increasing vegetable yield. In this paper, a novel approach was developed for the in-situ determination of the three-dimensional (3D) phenotype of vegetables by recording video clips using smartphones. First, a smartphone was used to record the vegetable from different angles, and then the key frame containing the crop area in the video was obtained using an algorithm based on the vegetation index and scale-invariant feature transform algorithm (SIFT) matching. After obtaining the key frame, a dense point cloud of the vegetables was reconstructed using the Structure from Motion (SfM) method, and then the segmented point cloud and a point cloud skeleton were obtained using the clustering algorithm. Finally, the plant height, leaf number, leaf length, leaf angle, and other phenotypic parameters were obtained through the point cloud and point cloud skeleton. Comparing the obtained phenotypic parameters to the manual measurement results, the root-mean-square error (RMSE) of the plant height, leaf number, leaf length, and leaf angle were 1.82, 1.57, 2.43, and 4.7, respectively. The measurement accuracy of each indicators is greater than 80%. The results show that the proposed method provides a convenient, fast, and low-cost 3D phenotype measurement pipeline. Compared to other methods based on photogrammetry, this method does not need a labor-intensive image-capturing process and can reconstruct a high-quality point cloud model by directly recording videos of crops.

Automatic Measurement of Morphological Traits of Typical Leaf Samples

“…The segmentation comparison of two widely used segmentation methods and our segmentation method is presented in Figure 12 . Method A is the Euclidean Clustering method in papers [ 24 , 29 ] and B is the Facet Region Growing method in paper [ 27 ]. It can be found out that the Euclidean Clustering method performs well in plants with no and a little canopy occlusion.…”

“…The two shape models, one using scale-related traits and the other using scale-invariant traits, are dynamically established based on the different segmentation results at different scales. The proposed segmentation method performs better compared with the Euclidean Clustering method in papers [ 24 , 29 ] and the Facet Region Growing method in paper [ 27 ]. Third, the time cost of our proposed method, especially the data processing time, is really low.…”

“…The 3D mesh model is reconstructed in real-time during data scanning and can be output directly, thus saving data processing time for 3D reconstruction in most contemporary research. Second, typical leaf samples can be automatically selected, and the accuracies of segmentation and trait estimation are competitive compared to related works [27,29,33]. A multi-level region growing segmentation method using two leaf shape models is proposed.…”

“… Segmentation results of two segmentation methods and our proposed method. Method A is the Euclidean Clustering method in papers [ 24 , 29 ] and B is the Facet Region Growing method in paper [ 27 ]. Plants in ( a ), ( b ), ( c ), and ( d ) are plants with no, a little, medium, and heavy canopy occlusion.…”

“…Specifically, leaves of plants, such as corn [22][23][24], rice [25], and wheat [26], are long and banding. Leaves of plants, such as Epipremnum aureum [27], Anthurium Andraeanum [28], and leafy vegetables [29], are broad and elliptical. The calculation algorithms of their morphological traits, especially length and width, are different.…”