FPCC: Fast point cloud clustering-based instance segmentation for industrial bin-picking

Xu, Yajun; Arai, Shigeo; Liu, Diyi; Lin, Fangzhou; Kosuge, Kazuhiro

doi:10.1016/j.neucom.2022.04.023

Cited by 21 publications

(7 citation statements)

References 78 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Algorithm 1: 3D Point visibility Algorithm Data: List of all Point cloud P(P x , P y , P z , P track id ), containing the points of the reference object reduce id ob j 3D Result: Disable all points except points in detected object point array for each point pts in P do if not pts track id in reduce id ob j 3D then do pts.valid = False Data: copy a new chunk and save to a new project The high confidence levels across the entire extent of instances lead to correct predictions. Different from modern approaches that predominantly rely on geometric clustering techniques [24] [25], our proposed approach does not require 3D-specific components, such as centre voting or manually tuned distance-based clustering. on the basis of the success of recent intersection bounded-box detection and 3D id-objects accumulation through bounded-box voting and grouping with the utilisation of 2D-to-3D bidirectional links, our overall 3D instance segmentation using 2D object detection yields promising results across a various of challenging archaeological sites and heritage documentation tasks, exploring an ancient shipwreck off Xlendi Bay, Gozo, is now publicly available on the Google Play Store app.…”

Section: Toward 3d Instance Segmentation Using Yolov4mentioning

confidence: 99%

Efficient 3D Instance Segmentation for Archaeological Sites Using 2D Object Detection and Tracking

kamal Al-anni,

DRAP

2024

IJCDS

View full text Add to dashboard Cite

This paper introduces an efficient method for 3D instance segmentation based on 2D object detection, applied to the photogrammetric survey images of archaeological sites. The method capitalizes on the relationship between the 3D model and the set of 2D images utilized to compute it. 2D detections on the images are projected and transformed into a 3D instance segmentation, thus identifying unique objects within the scene. The primary contribution of this work is the development of a semi-automatic image annotation method, augmented by an object tracking technique that leverages the temporal continuity of image sequences. Additionally, a novel ad-hoc evaluation process has been integrated into the conventional annotation-training-testing cycle to determine the necessity of additional annotations. This process tests the consistency of the 3D objects yielded by the 2D detection. The efficacy of the proposed method has been validated on the underwater site of Xlendi in Malta, resulting in complete and accurate 3D instance segmentation. Compared to traditional methods, the object tracking approach adopted has facilitated a 90% reduction in the need for manual annotations, The approach streamlines precise 3D detection, establishing a robust foundation for comprehensive 3D instance segmentation. This enhancement enriches the 3D survey, providing profound insights and facilitating seamless exploration of the Xlendi site from an archaeological perspective.

show abstract

Section: Toward 3d Instance Segmentation Using Yolov4mentioning

confidence: 99%

Efficient 3D Instance Segmentation for Archaeological Sites Using 2D Object Detection and Tracking

kamal Al-anni,

DRAP

2024

IJCDS

View full text Add to dashboard Cite

show abstract

“…Visual segmentation in cluttered scenes: robotic manipulation tasks are usually challenging due to the severe occlusion in dense clutters, and object segmentation in cluttered scenes has aroused extensive interest in robotic visual perception. Existing visual segmentation for densely cluttered objects can be categorized into two types: segmentation based on RGB-D images [35], [37], [36] and point cloud [12], [38]. For the first regard, robotic grasping [28], [39], [24], [40], [7] was usually guided by a visual segmentation module for the planner to generate the optimal grasp pose.…”

Section: Related Workmentioning

confidence: 99%

“…For visual segmentation methods based on the point cloud, Dong et al [12] extracted the point-wise features with the constraint that embedding of points from the same instance should be similar and vice versa, so that the clustered index in the feature space could be leveraged as the segmentation masks. Xu et al [38] inferred the geometric instance center via the learned point-wise features, and the remaining points were clustered into the closest center for segmentation. Nevertheless, the severe occlusion among objects cannot provide informative visual clues for accurate instance segmentation.…”

Section: Related Workmentioning

confidence: 99%

Category-level Shape Estimation for Densely Cluttered Objects

Wu¹,

Wang²,

Lu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Accurately estimating the shape of objects in dense clutters makes important contribution to robotic packing, because the optimal object arrangement requires the robot planner to acquire shape information of all existed objects. However, the objects for packing are usually piled in dense clutters with severe occlusion, and the object shape varies significantly across different instances for the same category. They respectively cause large object segmentation errors and inaccurate shape recovery on unseen instances, which both degrade the performance of shape estimation during deployment. In this paper, we propose a category-level shape estimation method for densely cluttered objects. Our framework partitions each object in the clutter via the multi-view visual information fusion to achieve high segmentation accuracy, and the instance shape is recovered by deforming the category templates with diverse geometric transformations to obtain strengthened generalization ability. Specifically, we first collect the multi-view RGB-D images of the object clutters for point cloud reconstruction. Then we fuse the feature maps representing the visual information of multi-view RGB images and the pixel affinity learned from the clutter point cloud, where the acquired instance segmentation masks of multi-view RGB images are projected to partition the clutter point cloud. Finally, the instance geometry information is obtained from the partially observed instance point cloud and the corresponding category template, and the deformation parameters regarding the template are predicted for shape estimation. Experiments in the simulated environment and real world show that our method achieves high shape estimation accuracy for densely cluttered everyday objects with various shapes.

show abstract

“…For point cloud segmentation, methods based on clustering [4,5], model fitting [6,7], machine learning [8], and deep learning [9,10] are mainly available. However, point cloud segmentation based on clustering methods is inefficient and requires a large amount of time and computational resources.…”

Section: Introductionmentioning

confidence: 99%

Arch dam point cloud segmentation based on deep feature learning and normal vector data optimization

Li,

et al. 2024

Preprint

View full text Add to dashboard Cite

Separating the dam body, spillway, and other structures from the point cloud in the dam area is an important step in dam deformation monitoring. Manual segmentation is time consuming and inaccurate. This study proposes a point cloud segmentation neural network model based on normal vector optimization suitable for dam environment: 1) This model utilizes the voxel uniform sampling method of equal length cubes to solve the problem of uneven point cloud density caused by wide range and long distance measurement during point cloud measurement in dam areas. 2) Designed block input and combined output modules in the model, achieving efficient input of large volume point cloud and eliminating the impact of interpolation points offset during seq2seq model decoding process. 3) In response to the diverse characteristics of point cloud normal vectors presented by vegetation, rock mass, and complex dam structures in the dam area, this paper proposes an adaptive radius plane fitting vector estimation method based on eigenvalue method to improve the accuracy of segmentation. Experiments on a prototype arch dam show that the proposed normal estimation method improves the classification accuracy of PointNet + + from the original 96.26–98.27%. Compared with the other three normal estimation methods (2-jets, Hough CNN, iterative PCA), the overall accuracy has improved by 0.82%, 1.22%, 0.22%, the mean intersection over union has improved by 0.0293, 0.0325, 0.0104. This study provides a high-precision classification scheme for applications such as dam deformation detection based on point cloud.

show abstract

FPCC: Fast point cloud clustering-based instance segmentation for industrial bin-picking

Cited by 21 publications

References 78 publications

Efficient 3D Instance Segmentation for Archaeological Sites Using 2D Object Detection and Tracking

Efficient 3D Instance Segmentation for Archaeological Sites Using 2D Object Detection and Tracking

Category-level Shape Estimation for Densely Cluttered Objects

Arch dam point cloud segmentation based on deep feature learning and normal vector data optimization

Contact Info

Product

Resources

About