Reconstruction method and optimum range of camera-shooting angle for 3D plant modeling using a multi-camera photography system

Abstract

Background: Measurement of plant structure is useful in monitoring plant conditions and understanding the responses of plants to environmental changes. 3D imaging technologies, especially the passive-SfM (Structure from Motion) algorithm combined with a multi-camera photography (MCP) system has been studied to measure plant structure due to its low-cost, close-range, and rapid image capturing ability. However, reconstruction of 3D plant models with complex structure is a time-consuming process and some systems have failed to reconstruct 3D models properly. Therefore, an MCP based SfM system was developed and an appropriate reconstruction method and optimal range of camera-shooting angles were investigated. Results: An MCP system which utilized 10 cameras and a rotary table for plant was developed. The 3D mesh model of a single leaf reconstruction using a set of images taken at each viewing zenith angle (VZA) from 12° (C2 camera) to 60° (C6 camera) by the MCP based SfM system had less undetected or unstable regions in comparison with other VZAs. The 3D mesh model of a whole plant, which merged 3D dense point cloud models built from a set of images taken at each appropriate VZA (Method 1), had high accuracy. The Method 1 error percentages for leaf area, leaf length, leaf width, stem height, and stem width are in the range of 2.6-4.4%, 0.2-2.2%, 1.0-4.9%, 1.9-2.8%, and 2.6-5.7% respectively. Also, the error of the leaf inclination angle was less than 5°. Conversely, the 3D mesh model of a whole plant built directly from a set of images taken at all appropriate VZAs (Method 2) had lower accuracy than that of Method 1. For Method 2, the error percentages of leaf area, leaf length, and leaf width are in the range of 3.1-13.3%, 0.4-3.3%, and 1.6-8.6%, respectively. It was difficult to obtain the error percentages of stem height and stem width because some information was missing in this model. In addition, the calculation time for Method 2 was 1.97 times longer computational time in comparison to Method 1. Conclusions: In this study, we determined the optimal shooting angles on the MCP based SfM system developed. We found that it is better in terms of computational time and accuracy to merge partial 3D models from images taken at each appropriate VZA, then construct complete 3D model (Method 1), rather than to construct 3D model by using images taken at all appropriate VZAs (Method 2). This is because utilization of incorporation of incomplete images to match feature points could result in reduced accuracy in 3D models and the increase in computational time for 3D model reconstruction.

Three-Dimensional Leaf Edge Reconstruction Combining Two- and Three-Dimensional Approaches

“…High-resolution 3D morphological data can be acquired efficiently and cost-effectively using light detection and ranging sensors, depth cameras, and photogrammetry techniques [ 39 – 41 ]. A pipeline utilizing structure from motion (SfM) and multiview stereo (MVS), which reconstructs a 3D surface as point cloud data from a series of 2D images captured from different angles, has been implemented in several libraries and software products (e.g., [ 42 , 43 ]).…”

A comparative study of plant phenotyping workflows based on three-dimensional reconstruction from multi-view images

“…A pipeline using structure from motion (SfM) [39,40] and multi-view stereo (MVS) [41,42], which reconstructs a 3D surface as point cloud data from a set of two-dimensional (2D) images captured from different angles, is widely available and implemented in several libraries and software products (e.g., [39,[42][43][44]). Based on the resultant high-resolution point cloud data, a three-dimensional morphological model can be constructed to yield phenotypic data (e.g., leaf number, plant height, and leaf area index (LAI) [45][46][47]).…”

Accuracy Evaluation and Branch Detection Method of 3D Modeling Using Backpack 3D Lidar SLAM and UAV-SfM for Peach Trees during the Pruning Period in Winter

“…UAV-SfM technology uses continuous and overlapping images obtained by a camera on the UAV to construct a 3D model. This method preserves the color information of the images in the 3D model, which makes it possible to be applied in different wavelengths, such as visible light, infrared, and thermal images [8][9][10][11][12][13][14]. Nevertheless, UAV-SfM technology requires more hardware resources and computing time to complete 3D matching and modeling.…”