Behind the Leaves: Estimation of Occluded Grapevine Berries With Conditional Generative Adversarial Networks

Abstract

The need for accurate yield estimates for viticulture is becoming more important due to increasing competition in the wine market worldwide. One of the most promising methods to estimate the harvest is berry counting, as it can be approached non-destructively, and its process can be automated. In this article, we present a method that addresses the challenge of occluded berries with leaves to obtain a more accurate estimate of the number of berries that will enable a better estimate of the harvest. We use generative adversarial networks, a deep learning-based approach that generates a highly probable scenario behind the leaves exploiting learned patterns from images with non-occluded berries. Our experiments show that the estimate of the number of berries after applying our method is closer to the manually counted reference. In contrast to applying a factor to the berry count, our approach better adapts to local conditions by directly involving the appearance of the visible berries. Furthermore, we show that our approach can identify which areas in the image should be changed by adding new berries without explicitly requiring information about hidden areas.

An RGB‐D object detection model with high‐generalization ability applied to tea harvesting robot for outdoor cross‐variety tea shoots detection

“…Bird et al (2021) generated lemon images using GANs and achieved higher image classification accuracy by enhancing training with synthetic images. Kierdorf et al (2022) employed Pix2Pix GAN to remove leaves from grape berry images and generate unobstructed grape berry images, enabling accurate grape berry counting. The second method, TL, involves loading a pretrained model on a large‐scale data set to quickly obtain a useful model for other object detection tasks, thus avoiding the issue of insufficient data leading to the inability of the model to converge properly.…”

“…generated lemon images using GANs and achieved higher image classification accuracy by enhancing training with synthetic images Kierdorf et al (2022). employed Pix2Pix GAN to remove leaves from grape berry images and generate unobstructed grape berry images, enabling accurate grape berry counting.…”

Learning Occluded Branch Depth Maps in Forest Environments Using RGB-D Images

“…While not dealing with occluded branches, in [20], a GAN was used to predict probable grayscale masks of occluded grapes. The networks were trained on masked grayscale images of manually exfoliated grapes which were then synthetically occluded.…”

Contrastive 3D Shape Completion and Reconstruction for Agricultural Robots Using RGB-D Frames

“…The main solutions for handling occlusions in agricultural environments have been constrained to 2D images, either by amodal instance segmentation methods that produce masks combining the visible and occluded part of crops [4] or by generative adversarial networks that estimate the appearance of crops as if there are no occlusions [16]. In contrast to the 2D case, fewer efforts have been made to tackle the occlusion problem in a 3D scene.…”