Segmentation and detection from organised 3D point clouds: a case study in broccoli head detection

Abstract

Autonomous harvesting is becoming an important challenge and necessity in agriculture, because of the lack of labour and the growth of population needing to be fed. Perception is a key aspect of autonomous harvesting and is very challenging due to difficult lighting conditions, limited sensing technologies, occlusions, plant growth, etc. 3D vision approaches can bring several benefits addressing the aforementioned challenges such as localisation, size estimation, occlusion handling and shape analysis. In this paper, we propose a novel approach using 3D information for detecting broccoli heads based on Convolutional Neural Networks (CNNs), exploiting the organised nature of the point clouds originating from the RGBD sensors. The proposed algorithm, tested on real-world datasets, achieves better performances than the state-of-the-art, with better accuracy and generalisation in unseen scenarios, whilst significantly reducing inference time, making it better suited for realtime in-field applications.

Deep Semantic Segmentation of 3D Plant Point Clouds

“…Future work should also explore alternative network architectures. Instead of using PointNet++, we are interested in one promising architecture, presented in the context of in-field broccoli head detection [9]. The authors take advantage of the fact that, as a result of the data acquisition technique, point clouds produced by some RGB-D sensors provide an organised structure and allow for the use of a CNN without need for projection.…”

Pheno4D: A spatio-temporal dataset of maize and tomato plant point clouds for phenotyping and advanced plant analysis

“…It requires learning from a dataset of manually labeled plant skeletons. Le Louedec et al [41] detects broccoli heads in point cloud data. However, they require the cloud data to be organized in an image-like grid which limits their use cases to only RGB-D sensors that capture the scene from a single point of view.…”

3D shape sensing and deep learning-based segmentation of strawberries

“…In previous work, we have demonstrated the suitability of using the standard CNN architectures in 3D crop detection systems for robotic harvesters Le Louedec et al (2020b) using points and surface normals, reducing the network size and using directly shape and spatial information easily computed without learning. Such idea can be found in Rabbani et al (2006), where local surface normals and curvature combined with points connectivity allows segmentation of point cloud parts.…”

“…A pure 3D crop detection system was proposed in Kusumam et al (2017a) for real-time detection of broccoli in the field which relied on hand-crafted features and SVM classifier. The suitability of using CNN architectures for the same application was later investigated in Le Louedec et al (2020b) with the superior results reported.…”