Counting of grapevine berries in images via semantic segmentation using convolutional neural networks

Zabawa, Laura; Kicherer, Anna; Klingbeil, Lasse; Töpfer, Reinhard; Kuhlmann, Heiner; Roscher, Ribana

doi:10.1016/j.isprsjprs.2020.04.002

Cited by 97 publications

(69 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although berry count has been successfully implemented by various authors (Aquino et al, 2018;Diago et al, 2015;Nuske et al, 2014), the logistical requirements for data acquisition at night can be complex. Interestingly, Zabawa et al, (2020) have presented a modern methodology which utilises a grape harvester refitted with sensors for counting grapevine berries, overcoming certain lighting limitations of previous studies (Font et al, 2015;Nuske et al, 2011). However, the modifications applied to the harvester should be considered an expensive alternative which will not always be practical, especially in developing countries.…”

Section: Yield Estimation: Plant-levelmentioning

confidence: 99%

“…Berry detection techniques (e.g. Grossetête et al, 2012;Nuske et al, 2014;Zabawa et al, 2020) generally bypass the pixellevel segmentation (bunch detection) and bunch metric approach. In this instance, individual berries are detected and counted for estimating the yield, commonly incorporating a historical berry mass during the estimation process (Nuske et al, 2014).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Vineyard yield estimation using 2-D proximal sensing: a multitemporal approach

2020

View full text Add to dashboard Cite

Vineyard yield estimation is a fundamental aspect in precision viticulture that enables a better understanding of the inherent variability within a vineyard. Yield estimation conducted early in the growing season provides insightful information to ensure the best fruit quality for the maximum desired yield. Proximal sensing techniques provide non-destructive in situ data acquisition for yield estimation during the growing season. This study aimed to determine the ideal phenological stage for yield estimation using 2-dimensional (2-D) proximal sensing and computer vision techniques in a vertical shoot positioned (VSP) vineyard. To achieve this aim, multitemporal digital imagery was acquired weekly over a 12-week period, with a final acquisition two days prior to harvest. Preceding the multitemporal analysis for yield estimation, an unsupervised k-means clustering (KMC) algorithm was evaluated for image segmentation on the final dataset captured before harvest, yielding bunch-level segmentation accuracies as high as 0.942, with a corresponding F1-score of 0.948. The segmentation yielded a pixel area (cm2), which served as input to a cross-validation model for calculating bunch mass (g). The ‘calculated mass’ was linearly regressed against the ‘actual mass’, indicating the capability for estimating vineyard yield. Results of the multitemporal analysis showed that the final stage of berry ripening was the ideal phenological stage for yield estimation, achieving a global r2 of 0.790.

show abstract

Section: Yield Estimation: Plant-levelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Vineyard yield estimation using 2-D proximal sensing: a multitemporal approach

2020

View full text Add to dashboard Cite

show abstract

“…The harvest estimation maps generated near the harvest time allow the zoning of harvest operations, reserving the most adequate yield levels, e.g., for premium wine production, as suggested by Ballesteros et al [ 25 ]. It does not need the creation of a training dataset for grape cluster detection, unlike most of the previously reported methodologies for fruit detection in field images [ 40 , 41 , 42 ]. It overcomes the problem of the on-ground image analysis related to the need of driving the image acquisition platform through the entire vineyard acquiring images from both sides of the vinerows, which in some cases is hard to achieve due to challenging conditions (e.g., slope, wet soil).…”

Section: Resultsmentioning

confidence: 99%

Grape Cluster Detection Using UAV Photogrammetric Point Clouds as a Low-Cost Tool for Yield Forecasting in Vineyards

Torres‐Sánchez

Mesas‐Carrascosa

Santesteban

et al. 2021

Sensors

View full text Add to dashboard Cite

Yield prediction is crucial for the management of harvest and scheduling wine production operations. Traditional yield prediction methods rely on manual sampling and are time-consuming, making it difficult to handle the intrinsic spatial variability of vineyards. There have been significant advances in automatic yield estimation in vineyards from on-ground imagery, but terrestrial platforms have some limitations since they can cause soil compaction and have problems on sloping and ploughed land. The analysis of photogrammetric point clouds generated with unmanned aerial vehicles (UAV) imagery has shown its potential in the characterization of woody crops, and the point color analysis has been used for the detection of flowers in almond trees. For these reasons, the main objective of this work was to develop an unsupervised and automated workflow for detection of grape clusters in red grapevine varieties using UAV photogrammetric point clouds and color indices. As leaf occlusion is recognized as a major challenge in fruit detection, the influence of partial leaf removal in the accuracy of the workflow was assessed. UAV flights were performed over two commercial vineyards with different grape varieties in 2019 and 2020, and the photogrammetric point clouds generated from these flights were analyzed using an automatic and unsupervised algorithm developed using free software. The proposed methodology achieved R2 values higher than 0.75 between the harvest weight and the projected area of the points classified as grapes in vines when partial two-sided removal treatment, and an R2 of 0.82 was achieved in one of the datasets for vines with untouched full canopy. The accuracy achieved in grape detection opens the door to yield prediction in red grape vineyards. This would allow the creation of yield estimation maps that will ease the implementation of precision viticulture practices. To the authors’ knowledge, this is the first time that UAV photogrammetric point clouds have been used for grape clusters detection.

show abstract

“…Unlike scene recognition (i.e., produce one label for one image patch), deep learning models for semantic segmentation aim to provide dense labels in a patch (Long et al 2015;Chen et al 2017;Ronneberger et al 2015). The advantage of such deep-learning-based semantic segmentation models has attracted the remote sensing community's attention for delineating the boundaries of trees (Zabawa et al 2020), fields (Waldner and Diakogiannis 2020), sparse settlements , roads (Zhang, Liu, et al 2018), and street view objects (Fang and Lafarge 2019). These applications were primarily made with very high-resolution satellite images, UAV images, point clouds, or at least 20m Sentinel-2 images.…”

Section: Introductionmentioning

confidence: 99%

Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: A semantic segmentation solution

Chen

Qiu

Schmitt

et al. 2020

Remote Sensing of Environment

View full text Add to dashboard Cite

Landsat imagery is an unparalleled freely available data source that allows reconstructing land-cover and land-use change, including horizontal and vertical urban form. This paper addresses the challenge of using Landsat data, particularly its 30m spatial resolution, for monitoring three-dimensional urban densification. Unlike conventional convolutional neural networks (CNNs) for scene recognition resulting in resolution loss, the proposed semantic segmentation framework provides a pixel-wise classification and improves the accuracy of urban form mapping. We compare temporal and spatial transferability of an adapted DeepLab model with a simple fully convolutional network (FCN) and a texture-based random forest (RF) model to map urban density in the two morphological dimensions: horizontal (compact, open, sparse) and vertical (high rise, low rise). We test whether a model trained on the 2014 data can be applied to 2006 and 1995 for Denmark, and examine whether we could use the model trained on the Danish data to accurately map ten other European cities. Our results show that an implementation of deep networks and the inclusion of multi-scale contextual information greatly improve the classification and the model's ability to generalize across space and time. Between the two semantic segmentation models, DeepLab provides more accurate horizontal and vertical classifications than FCN when sufficient training data is available. By using DeepLab, the F1 score can be increased by 4 and 10 percentage points for detecting vertical urban growth compared to FCN and RF for Denmark. For mapping the ten other European cities with training data from Denmark, DeepLab also shows an advantage of 6 percentage points over RF for both horizontal and vertical dimensions. The resulting maps across the years 1985 to 2018 reveal different patterns of urban growth between Copenhagen and Aarhus, the two largest cities in Denmark, illustrating that those cities have used various planning policies in addressing population growth and housing supply challenges. In summary, we propose a transferable deep learning approach for automated, long-term mapping of urban form from Landsat images that is effective in areas experiencing a slow pace of urban growth or with small-scale changes.

show abstract

Counting of grapevine berries in images via semantic segmentation using convolutional neural networks

Cited by 97 publications

References 30 publications

Vineyard yield estimation using 2-D proximal sensing: a multitemporal approach

Vineyard yield estimation using 2-D proximal sensing: a multitemporal approach

Grape Cluster Detection Using UAV Photogrammetric Point Clouds as a Low-Cost Tool for Yield Forecasting in Vineyards

Mapping horizontal and vertical urban densification in Denmark with Landsat time-series from 1985 to 2018: A semantic segmentation solution

Contact Info

Product

Resources

About