Three-Dimensional Reconstruction of Soybean Canopies Using Multisource Imaging for Phenotyping Analysis

Abstract

Geometric three-dimensional (3D) reconstruction has emerged as a powerful tool for plant phenotyping and plant breeding. Although laser scanning is one of the most intensely used sensing techniques for 3D reconstruction projects, it still has many limitations, such as the high investment cost. To overcome such limitations, in the present study, a low-cost, novel, and efficient imaging system consisting of a red-green-blue (RGB) camera and a photonic mixer detector (PMD) was developed, and its usability for plant phenotyping was demonstrated via a 3D reconstruction of a soybean plant that contains color information. To reconstruct soybean canopies, a density-based spatial clustering of applications with noise (DBSCAN) algorithm was used to extract canopy information from the raw 3D point cloud. Principal component analysis (PCA) and iterative closest point (ICP) algorithms were then used to register the multisource images for the 3D reconstruction of a soybean plant from both the side and top views. We then assessed phenotypic traits such as plant height and the greenness index based on the deviations of test samples. The results showed that compared with manual measurements, the side view-based assessments yielded a determination coefficient (R2) of 0.9890 for the estimation of soybean height and a R2 of 0.6059 for the estimation of soybean canopy greenness index; the top view-based assessment yielded a R2 of 0.9936 for the estimation of soybean height and a R2 of 0.8864 for the estimation of soybean canopy greenness. Together, the results indicated that an assembled 3D imaging device applying the algorithms developed in this study could be used as a reliable and robust platform for plant phenotyping, and potentially for automated and high-throughput applications under both natural light and indoor conditions.

Estimating 3D Chlorophyll Content Distribution of Trees Using an Image Fusion Method Between 2D Camera and 3D Portable Scanning Lidar

“…Novelty and usefulness of the present fusion method emerge through comparison between the previous methods and the present one. Some of the previous fusion systems between the lidar and 2D images were designed and assembled for a specific 2D camera and lidar, where the positional relation of the 2D camera and lidar was fixed by frames or other members [26,27]. In these systems, other 2D cameras and lidars are difficult to use at the same fixed arrangement and settings because the positional relation needs to be changed depending on the size or form of the camera and lidar, and according to image properties, such as resolution, view angle, and distortion.…”

“…Portable lidar images and 2D thermal, spectral, and Chl fluorescence images were fused in small plants and 3D plant responses to herbicide treatments were clearly shown [24,25]. The fusion of an RGB camera and depth camera (one of the lidar instruments) was used for soybean canopy analysis [26]. The image fusion was conducted by a portable lidar and thermal camera to obtain plant spatial temperature distribution [27].…”

“…To apply the fusion method to various types and sizes of plants, a method is required in which the 2D image is directly fused to the 3D point cloud image without conversion to the polygonal surface mesh. In the studies by [16,26], fusion systems were designed and assembled for specific 2D cameras and lidars. To effectively obtain plant physiological and biochemical information together with the structural features, the fusion method should be easily applicable to different types of 2D cameras and lidars.…”

Image-Based Dynamic Quantification of Aboveground Structure of Sugar Beet in Field

“…Plant height, as a basic phenotypic trait, can be used not only as an intuitive indicator of plant growth state, but also for the estimation of plant biomass and yield [44,45]. In this study, the ground truth of plant height was measured as the vertical distance between the bases of the stem to the highest point of the plant.…”

High-Throughput Phenotyping Analysis of Potted Soybean Plants Using Colorized Depth Images Based on A Proximal Platform

“…In addition, changes in light intensity affected the results of 3D reconstruction, which in turn decreased the accuracy of plant height calculation. Time-of-flight (ToF) cameras have been proven to be useful image acquisition devices for phenotyping research based on vertical top-view and side-view measurements of a plant canopy [33]. 3D cloud points can be obtained with a very high frame rate (40 frames/s); however, their resolution is relatively low (200 × 200 pixels), which restricts the capacity of high-throughput phenotyping analysis.…”