Automatic Recognition of Ripening Tomatoes by Combining Multi-Feature Fusion with a Bi-Layer Classification Strategy for Harvesting Robots

Abstract

Automatic recognition of ripening tomatoes is a main hurdle precluding the replacement of manual labour by robotic harvesting. In this paper, we present a novel automatic algorithm for recognition of ripening tomatoes using an improved method that combines multiple features, feature analysis and selection, a weighted relevance vector machine (RVM) classifier, and a bi-layer classification strategy. The algorithm operates using a two-layer strategy. The first-layer classification strategy aims to identify tomato-containing regions in images using the colour difference information. The second classification strategy is based on a classifier that is trained on multi-medium features. In our proposed algorithm, to simplify the calculation and to improve the recognition efficiency, the processed images are divided into 9 × 9 pixel blocks, and these blocks, rather than single pixels, are considered as the basic units in the classification task. Six colour-related features, namely the Red (R), Green (G), Blue (B), Hue (H), Saturation (S) and Intensity (I) components, respectively, colour components, and five textural features (entropy, energy, correlation, inertial moment and local smoothing) were extracted from pixel blocks. Relevant features and their weights were analysed using the iterative RELIEF (I-RELIEF) algorithm. The image blocks were classified into different categories using a weighted RVM classifier based on the selected relevant features. The final results of tomato recognition were determined by combining the block classification results and the bi-layer classification strategy. The algorithm demonstrated the detection accuracy of 94.90% on 120 images, this suggests that the proposed algorithm is effective and suitable for tomato detection

Automatic Detection of Single Ripe Tomato on Plant Combining Faster R-CNN and Intuitionistic Fuzzy Set

“…In a previous fruit segmentation method, Wei et al [1] used a combined OHTA color space and Otsu threshold algorithm to segment mature fruit with high accuracy; however, that approach cannot handle occluded fruit. Wu et al [10] developed a multi-feature fusion method that included the iterative RELIEF (I-RELIEF) algorithm, a weighted relevance vector machine (RVM) classifier, and a two-layer classification strategy to recognize ripening tomatoes. Sa et al [20] adopted a state-of-the-art object detector termed the Faster Region-based CNN (Faster R-CNN) model and combined color (RGB) with near-infrared (NIR) information, which led to a novel multi-model Faster R-CNN and achieved good results in fruit detection.…”

“…Hence, multifeature information fusion was adopted by researchers, which used both the color feature and the shape feature to improve the recognition rate of fruits. Wu et al [10] developed an improved method that combined multiple features, feature analysis and selection, a weighted relevance vector machine (RVM) classifier, and a bilayer classification strategy to recognize ripening tomatoes. Yamamoto et al [11] proposed an image processing method for accurately detecting individual intact tomatoes on plants, including mature, immature and young fruits, using a conventional RGB digital camera in conjunction with machine learning approaches.…”

A Literature Review on Machine Vision based Approaches for Ripeness Detection of Fruits

“…In this paper, Wu et al [1] proposed a novel automatic algorithm for recognizing tomatoes for ripening by the help of an improved method that merges multiple features, feature analysis and selection, a weighted relevance vector machine (RVM classifier) and bi-layer classifier. The algorithm undergoes two levels of classification.…”

Recognition of Bloom/Yield in Crop Images Using Deep Learning Models for Smart Agriculture: A Review

“…Crop disease detection can utilize simple linear iterative clustering features to segment the super pixels in a CIELAB color model [89]. The autonomous maturity detection of tomato berries was developed with the fusion of multiple (colortexture) features using an iterative RELIEF algorithm [90]. Principal component analysis (PCA), a pixel level classification technique, could automatically detect diseases in pepper leaves from the color features [91].…”