Field phenotyping of grapevine growth using dense stereo reconstruction

Abstract

BackgroundThe demand for high-throughput and objective phenotyping in plant research has been increasing during the last years due to large experimental sites. Sensor-based, non-invasive and automated processes are needed to overcome the phenotypic bottleneck, which limits data volumes on account of manual evaluations. A major challenge for sensor-based phenotyping in vineyards is the distinction between the grapevine in the foreground and the field in the background – this is especially the case for red-green-blue (RGB) images, where similar color distributions occur both in the foreground plant and in the field and background plants. However, RGB cameras are a suitable tool in the field because they provide high-resolution data at fast acquisition rates with robustness to outdoor illumination.ResultsThis study presents a method to segment the phenotypic classes ‘leaf’, ‘stem’, ‘grape’ and ‘background’ in RGB images that were taken with a standard consumer camera in vineyards. Background subtraction is achieved by taking two images of each plant for depth reconstruction. The color information is furthermore used to distinguish the leaves from stem and grapes in the foreground. The presented approach allows for objective computation of phenotypic traits like 3D leaf surface areas and fruit-to-leaf ratios. The method has been successfully applied to objective assessment of growth habits of new breeding lines. To this end, leaf areas of two breeding lines were monitored and compared with traditional cultivars. A statistical analysis of the method shows a significant (p <0.001) determination coefficient R 2= 0.93 and root-mean-square error of 3.0%.ConclusionsThe presented approach allows for non-invasive, fast and objective assessment of plant growth. The main contributions of this study are 1) the robust segmentation of RGB images taken from a standard consumer camera directly in the field, 2) in particular, the robust background subtraction via reconstruction of dense depth maps, and 3) phenotypic applications to monitoring of plant growth and computation of fruit-to-leaf ratios in 3D. This advance provides a promising tool for high-throughput, automated image acquisition, e.g., for field robots.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0560-x) contains supplementary material, which is available to authorized users.

Automatic image-based determination of pruning mass as a determinant for yield potential in grapevine management and breeding

“…Two monochrome images and one RGB image per vine (MCS) were captured with real background. The two monochrome images were rectified in a pre‐processing step as described by Klodt et al (). The rectified images had parallel epipolar lines that allowed for efficient computation of depth maps.…”

“…Surface areas were computed using the dense depth reconstruction for all points in the foreground class (Figure ). The complete method is described by Klodt et al ().…”

“…The use of cameras as optical sensors facilitates fast, low‐cost and robust data recording in the field (Herzog et al ). To avoid the use of an artificial background, three‐dimensional stereo reconstruction and calculation of depth maps are suitable (Klodt et al ). Here, we describe a non‐invasive image‐based method to predict the PM out of images by using automated depth segmentation and minor user interactions.…”

Machine Vision System for 3D Plant Phenotyping

“…Paulus et al [20], [21] showed a feature based histogram analysis method to classify different organs in wheat, grapevine and barley plants. A segmentation algorithm to monitor grapevine growth was presented by Klodt et al [22]. A similar type of work on plant organ segmentation by unsupervised clustering was proposed by Wahabzada et al [23].…”

Natural Inspired Intelligent Visual Computing and Its Application to Viticulture

“…The disadvantage of using remote sensing technologies is the high costs that are required. More recently, authors have proposed using digital imaging and computer vision techniques using cheaper ground-based cameras [22]. The work by Rodríguez-Pulido et al [23] described a characterization of grape seeds and grape berries by digital image analysis and showed that the development of the grape could be visually determined by tracing its changes in size and color.…”