Towards a generalized colour image segmentation for kiwifruit detection

Abstract

Abstract-Developing robust computer vision algorithms to detect fruit in trees is challenging due to less controllable conditions, including variation in illumination within an image as well as between image sets. There are two classes of techniques: local-feature-based techniques and shape-based techniques, which have been used extensively in this application domain. Out of the two classes, the local-feature-based techniques have shown higher accuracies over shape-based techniques, but are less desirable due to the requirement of repeated calibration. In this paper, we investigate the potential of developing a generalized colour pixel classifier that can be employed to detect kiwifruit on vines, under variable fruit maturity levels and imaging conditions. First, we observed the colour data patterns of fruit and nonfruit regions from different image sets. With consistant data patterns it was found that a suitable normalization could produce an invariant colour descriptor. Then, a neural network Self-Organizing Map (SOM) model, which has a hierarchical clustering ability was used to investigate the potential of developing a generalized neural network model to classify pixels under variable conditions. Models were built for colour features extracted in CIELab space for both absolute colour values and relative colour descriptors. The paper presents the positive results of the preliminary investigations. The conditions for a successful application of the approach as well as the potential for extending it for automatic calibration will also be discussed.

Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

“…Image processing at orchards spans a large variety of fruits such as grapes (Nuske et al, 2014;Font et al, 2015), mangoes (Chhabra et al, 2012;, apples (Linker et al, 2012;Wang et al, 2013;Stajnko et al, 2009;Silwal et al, 2014;Ji et al, 2012;Kim et al, 2015;Hung et al, 2015), citrus (Li et al, 2011;Annamalai et al, 2004;Sengupta and Lee, 2014;Regunathan and Lee, 2005;Qiang et al, 2014), kiwifruit (Wijethunga et al, 2009) and peaches (Kurtulmus et al, 2014). Fruit classification is generally performed by transforming image regions into discriminative feature spaces and using a trained classifier to associate them to either fruit regions or background objects such as foliage, branches, ground etc.…”

Kiwifruit detection in challenging conditions

“…In contrast, learning methods are robust to dynamic conditions such as lighting, camera viewpoint, fruit occlusion and fruit size. Wijethunga et al [Wijethunga et al, 2009] used a Self-Organizing Map (SOM) model to detect kiwifruit. The normalised pixel data in L*a*b* colour space was used for training the model, and the author concluded that its performance was better compared to using the original dataalthough the author has not quantified the result.…”

“…Jimenez et al [A. R. Jiménez et al, 2000] suggested that using artificial lighting can reduce shadows caused by sunlight. Another suggestion, for having consistent lighting conditions, was capturing images at nighttime [Fu et al, 2015;Fu et al, 2017;Wijethunga et al, 2009;Wang et al, 2013] or on an overcast day [Dias et al, 2018].…”