Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions

Palacios, Fernando Fernández; Bueno, Gloria; Jesus, S. M.; Diago, María P.; Hernández, Inés; Tardáguila, Javier

doi:10.1016/j.compag.2020.105796

Cited by 52 publications

(34 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yield assessment can also be beneficed by this practice to reduce the fruit occlusion affected by leaves. The image acquisition could be carried out on-the-go by mobile sensing platforms moving at conventional tractor speed in in vineyards trained to a vertical shoot position (VSP) system [8,17], allowing for a rapid image processing for determining leaf occlusion rate and/or yield. The next steps towards the automation of vineyard canopy assessment using new proximal sensors were recently shown; a new system equipped with matrix-based optical RGB sensors was mounted in a tractor to assess the leaf layer number and vineyard canopy gaps [28].…”

Section: Figurementioning

confidence: 99%

See 1 more Smart Citation

Impact of Leaf Occlusions on Yield Assessment by Computer Vision in Commercial Vineyards

et al. 2021

Self Cite

View full text Add to dashboard Cite

Yield assessment has been identified as critical topic for grape and wine industry. Computer vision has been applied for assessing yield, but the accuracy was greatly affected by fruit occlusion affected by leaves and other plant organs. The objective of this work was the consistent, continuous evaluation of the impact of leaf occlusions in different commercial vineyard plots at different defoliation stages. RGB (red, green and blue) images from five Tempranillo (Vitis vinifera L.) vineyards were manually acquired using a digital camera under field conditions at three different levels of defoliation: no defoliation, partial defoliation and full defoliation. Computer vision was used for the automatic detection of different canopy features, and for the calibration of regression equations for the prediction of yield computed per vine segment. Leaf occlusion rate (berry occlusion affected by leaves) was computed by machine vision in no defoliated vineyards. As occlusion rate increased, R2 between bunch pixels and yield was gradually reduced, ranging from 0.77 in low occlusion, to 0.63.

show abstract

Section: Figurementioning

confidence: 99%

“…Image analysis has been widely applied in viticulture for assessing crop yield [9][10][11][12][13][14][15]. Yield forecasting has been carried out at different phenological stages: Budburst [16], flowering [17], pea-size [10,13] and harvest [10,12]. Most of the previous works have focused on visible fruits.…”

Section: Introductionmentioning

confidence: 99%

Impact of Leaf Occlusions on Yield Assessment by Computer Vision in Commercial Vineyards

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this work, this decrease in detection performance was accepted due to the high gain in runtime performance. In comparison with state-of-the-art works such as the one proposed by Palacios et al [30], which detected grape flower at the bloom with an F1 score of 73.0%, this work presented a lower detection performance. On the other hand, the tradeoff between detection performance and runtime performance was extremely satisfactory since, especially for SSD MobileNet-V1, the model could perform with a mAP up to 66.96%, performing the detection at a rate higher than 10 Hz per image.…”

Section: Discussionmentioning

confidence: 61%

“…In this work, the grape bunches were placed over uniform backgrounds and were separated from each other by the application of a threshold. Palacios et al [30] presented a DL-based approach where the region of interest containing aggregations of flowers was extracted using a semantic segmentation architecture. For the detection of grape bunches at more advanced growth stages, Pérez-Zavala et al [31] clustered pixels into grape bunches using shape and texture information from images.…”

Section: Related Workmentioning

confidence: 99%

Grape Bunch Detection at Different Growth Stages Using Deep Learning Quantized Models

et al. 2021

View full text Add to dashboard Cite

The agricultural sector plays a fundamental role in our society, where it is increasingly important to automate processes, which can generate beneficial impacts in the productivity and quality of products. Perception and computer vision approaches can be fundamental in the implementation of robotics in agriculture. In particular, deep learning can be used for image classification or object detection, endowing machines with the capability to perform operations in the agriculture context. In this work, deep learning was used for the detection of grape bunches in vineyards considering different growth stages : the early stage just after the bloom and the medium stage where the grape bunches present an intermediate development. Two state-of-the-art single-shot multibox models were trained, quantized, and deployed in a low-cost and low-power hardware device, a Tensor Processing Unit. The training input was a novel and publicly available dataset proposed in this work. This dataset contains 1929 images and respective annotations of grape bunches at two different growth stages, captured by different cameras in several illumination conditions. The models were benchmarked and characterized considering the variation of two different parameters: the confidence score and the intersection over union threshold. The results showed that the deployed models could detect grape bunches in images with a medium average precision up to 66.96%. Since this approach uses low resources, a low-cost and low-power hardware device that requires simplified models with 8 bit quantization, the obtained performance was satisfactory. Experiments also demonstrated that the models performed better in identifying grape bunches at the medium growth stage, in comparison with grape bunches present in the vineyard after the bloom, since the second class represents smaller grape bunches, with a color and texture more similar to the surrounding foliage, which complicates their detection.

show abstract

“…The standard or traditional methods retrieve limited data and produce a static prediction in a multi-step process of determining average number of clusters per vine, number of berries per cluster, and weight per cluster or berry with the growth overall 10% error greatly dependent on adequate staffing and extensive historical databases of cluster weights and yields [37] Computer vision and image processing are leading the alternative methods and are one of the most utilized techniques for attempting an early yield estimation. Still, different approaches such as Synthetic Aperture Radar (SAR), low frequency ultrasound [38], RF Signals [39], counting number of flowers [40][41][42][43][44][45][46][47], Boolean model application [48], shoot count [49], shoot biomass [50,51], frequency-modulated continuous-wave (FMCW) radar [52,53], detection of specular spherical reflection peaks [54], the combination of RGB and multispectral imagery [55] along with derived occlusion ratios, are alternative methods.…”

Section: Resultsmentioning

confidence: 99%

Vineyard Yield Estimation, Prediction, and Forecasting: A Systematic Literature Review

2021

View full text Add to dashboard Cite

Purpose—knowing in advance vineyard yield is a critical success factor so growers and winemakers can achieve the best balance between vegetative and reproductive growth. It is also essential for planning and regulatory purposes at the regional level. Estimation errors are mainly due to the high inter-annual and spatial variability and inadequate or poor performance sampling methods; therefore, improved applied methodologies are needed at different spatial scales. This paper aims to identify the alternatives to traditional estimation methods. Design/methodology/approach—this study consists of a systematic literature review of academic articles indexed on four databases collected based on multiple query strings conducted on title, abstract, and keywords. The articles were reviewed based on the research topic, methodology, data requirements, practical application, and scale using PRISMA as a guideline. Findings—the methodological approaches for yield estimation based on indirect methods are primarily applicable at a small scale and can provide better estimates than the traditional manual sampling. Nevertheless, most of these approaches are still in the research domain and lack practical applicability in real vineyards by the actual farmers. They mainly depend on computer vision and image processing algorithms, data-driven models based on vegetation indices and pollen data, and on relating climate, soil, vegetation, and crop management variables that can support dynamic crop simulation models. Research limitations—this work is based on academic articles published before June 2021. Therefore, scientific outputs published after this date are not included. Originality/value—this study contributes to perceiving the approaches for estimating vineyard yield and identifying research gaps for future developments, and supporting a future research agenda on this topic. To the best of the authors’ knowledge, it is the first systematic literature review fully dedicated to vineyard yield estimation, prediction, and forecasting methods.

show abstract

Automated grapevine flower detection and quantification method based on computer vision and deep learning from on-the-go imaging using a mobile sensing platform under field conditions

Cited by 52 publications

References 26 publications

Impact of Leaf Occlusions on Yield Assessment by Computer Vision in Commercial Vineyards

Impact of Leaf Occlusions on Yield Assessment by Computer Vision in Commercial Vineyards

Grape Bunch Detection at Different Growth Stages Using Deep Learning Quantized Models

Vineyard Yield Estimation, Prediction, and Forecasting: A Systematic Literature Review

Contact Info

Product

Resources

About