Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping

Lin, Yunzhi; Tang, Chao; Chu, Fu-Jen; Vela, Patricio A.

doi:10.1109/icra40945.2020.9197256

Cited by 36 publications

(29 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1.0.3 Contribution. This study extends a previous conference version (Lin et al 2020) beyond proof-of-concept and focuses on the value of explicitly encoding shape information. It improves the dataset generation strategy and replaces the post-processing components downstream of shape segmentation with improved model-based approaches based on best practice.…”

Section: Introductionmentioning

confidence: 84%

Primitive Shape Recognition for Object Grasping

Lin¹,

Tang²,

Chu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Shape informs how an object should be grasped, both in terms of where and how. As such, this paper describes a segmentation-based architecture for decomposing objects sensed with a depth camera into multiple primitive shapes, along with a post-processing pipeline for robotic grasping. Segmentation employs a deep network, called PS-CNN, trained on synthetic data with 6 classes of primitive shapes and generated using a simulation engine. Each primitive shape is designed with parametrized grasp families, permitting the pipeline to identify multiple grasp candidates per shape region. The grasps are rank ordered, with the first feasible one chosen for execution. For task-free grasping of individual objects, the method achieves a 94.2% success rate placing it amongst the top performing grasp methods when compared to top-down and SE(3)-based approaches. Additional tests involving variable viewpoints and clutter demonstrate robustness to setup. For task-oriented grasping, PS-CNN achieves a 93.0% success rate. Overall, the outcomes support the hypothesis that explicitly encoding shape primitives within a grasping pipeline should boost grasping performance, including task-free and task-relevant grasp prediction.

show abstract

Section: Introductionmentioning

confidence: 84%

Primitive Shape Recognition for Object Grasping

Lin¹,

Tang²,

Chu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A sufficient amount of data is required to achieve high-performance deep neural network, yet it is labor-intensive and expensive. Therefore, various previous studies have improved performance by using synthetic data when sufficient amounts of data are unavailable [44][45][46][47][48][49]. Thus, we generated 37,323 synthetic datasets and used them to achieve high performance.…”

Section: Discussionmentioning

confidence: 99%

Deep Learning Based Detection of Missing Tooth Regions for Dental Implant Planning in Panoramic Radiographic Images

et al. 2022

View full text Add to dashboard Cite

Dental implantation is a surgical procedure in oral and maxillofacial surgery. Detecting missing tooth regions is essential for planning dental implant placement. This study proposes an automated method that detects regions of missing teeth in panoramic radiographic images. Tooth instance segmentation is required to accurately detect a missing tooth region in panoramic radiographic images containing obstacles, such as dental appliances or restoration. Therefore, we constructed a dataset that contains 455 panoramic radiographic images and annotations for tooth instance segmentation and missing tooth region detection. First, the segmentation model segments teeth into the panoramic radiographic image and generates teeth masks. Second, a detection model uses the teeth masks as input to predict regions of missing teeth. Finally, the detection model identifies the position and number of missing teeth in the panoramic radiographic image. We achieved 92.14% mean Average Precision (mAP) for tooth instance segmentation and 59.09% mAP for missing tooth regions detection. As a result, this method assists diagnosis by clinicians to detect missing teeth regions for implant placement.

show abstract

“…The first type mainly employs geometric information to approximate the object or robot gripper for 3-D grasp detection. Requireing prior object knowledge for candidate grasp generation, model-based methods approximated object with box-based shapes (Huebner and Kragic 2008) or primitive surface shapes (Lin et al 2020). Once the shape is recognized, extracted, and estimated, the grasps naturally follow.…”

Section: D Grasp Predictionmentioning

confidence: 99%

“…Analysis of Outcomes Using the confidence interval of GKNet as a starting point for the comparison, only two methods score higher than the lower limit of the confidence interval (92.9%), Viereck et al (2017) and Lin et al (2020).…”

Section: Comparison Of Grasping Outcomes With Published Workmentioning

confidence: 99%

GKNet: grasp keypoint network for grasp candidates detection

Xu¹,

Chu²,

Vela³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Contemporary grasp detection approaches employ deep learning to achieve robustness to sensor and object model uncertainty. The two dominant approaches design either grasp-quality scoring or anchor-based grasp recognition networks. This paper presents a different approach to grasp detection by treating it as keypoint detection. The deep network detects each grasp candidate as a pair of keypoints, convertible to the grasp representation g = {x, y, w, θ} T , rather than a triplet or quartet of corner points. Decreasing the detection difficulty by grouping keypoints into pairs boosts performance. To further promote dependencies between keypoints, the general non-local module is incorporated into the proposed learning framework. A final filtering strategy based on discrete and continuous orientation prediction removes false correspondences and further improves grasp detection performance. GKNet, the approach presented here, achieves the best balance of accuracy and speed on the Cornell and the abridged Jacquard dataset (96. 9% and 98.39% at 41.67 and 23.26 fps). Follow-up experiments on a manipulator evaluate GKNet using 4 types of grasping experiments reflecting different nuisance sources: static grasping, dynamic grasping, grasping at varied camera angles, and bin picking. GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and bin picking experiments. The results confirm the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robustness to expected nuisance factors.

show abstract

Using Synthetic Data and Deep Networks to Recognize Primitive Shapes for Object Grasping

Cited by 36 publications

References 40 publications

Primitive Shape Recognition for Object Grasping

Primitive Shape Recognition for Object Grasping

Deep Learning Based Detection of Missing Tooth Regions for Dental Implant Planning in Panoramic Radiographic Images

GKNet: grasp keypoint network for grasp candidates detection

Contact Info

Product

Resources

About