SKP: Semantic 3D Keypoint Detection for Category-Level Robotic Manipulation

Luo, Zhong‐Zhen; Xue, Wanqi; Chae, Julia; Fu, Geng

doi:10.1109/lra.2022.3157438

Cited by 9 publications

(2 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D reconstruction technology mainly obtains data information such as laser point cloud and images through laser scanning or cameras, then analyzes and processes the obtained data information, and uses 3D reconstruction related methods to model and reproduce scenes in the real world. It has been widely used in the fields of 3D real scene [1][2][3], digital twin [4][5][6], virtual reality [7,8], artificial intelligence [9][10][11], automatic driving [12][13][14], Indoor Robots [15][16][17][18], Outdoor Robots [19][20][21][22][23], UAV Applications [24][25][26][27], 3D printing [28] and so on.…”

Section: Introductionmentioning

confidence: 99%

Overview of image-based 3D reconstruction technology

Niu,

Liu,

Huang

et al. 2024

J. Eur. Opt. Society-Rapid Publ.

View full text Add to dashboard Cite

Three-dimensional(3D) reconstruction technology is the key technology to establish and express the objective world by using computer, and it is widely used in real 3D, automatic driving, aerospace, navigation and industrial robot applications. According to different principles, it is mainly divided into methods based on traditional multi-view geometry and methods based on deep learning. This paper introduces the above methods from the perspective of three-dimensional space representation. The feature extraction and stereo matching theory of traditional 3D reconstruction methods are the theoretical basis of 3D reconstruction methods based on deep learning, so the paper focuses on them. With the development of traditional 3D reconstruction methods and the development of deep learning related theories, the explicit deep learning 3D reconstruction method represented by MVSNet and the implicit 3D reconstruction method represented by NeRF have been gradually developed. At the same time, the dataset and evaluation indicators for 3D reconstruction were introduced. Finally, a summary of image based 3D reconstruction was provided. Deep networks based on deep learning have been widely used in computer vision, especially the application of deep learning in depth networks of depth estimation, which will eventually realize real-time pixel-level reconstruction of 3D scenes at different scales.

show abstract

Section: Introductionmentioning

confidence: 99%

Overview of image-based 3D reconstruction technology

Niu,

Liu,

Huang

et al. 2024

J. Eur. Opt. Society-Rapid Publ.

View full text Add to dashboard Cite

show abstract

“…Existing work on specifying task-specific constraints and regions for grasping objects use data-driven methods to learn from simulation trials or to define task-specific approach vectors [2]. There is also work on associating a semantic keypoint skeleton to an object class to identify grasp poses for specific object classes [3,4]. However, existing methods require many labelled examples and do not encode constraints that jointly consider different tasks, objects, and grasp poses.…”

Section: Introductionmentioning

confidence: 99%

Generating Task-specific Robotic Grasps

Sridharan¹

2022

Preprint

View full text Add to dashboard Cite

This paper describes a method for generating robot grasps by jointly considering stability and other task and object-specific constraints. We introduce a three-level representation that is acquired for each object class from a small number of exemplars of objects, tasks, and relevant grasps. The representation encodes task-specific knowledge for each object class as a relationship between a keypoint skeleton and suitable grasp points that is preserved despite intra-class variations in scale and orientation. The learned models are queried at run time by a simple sampling-based method to guide the generation of grasps that balance task and stability constraints. We ground and evaluate our method in the context of a Franka Emika Panda robot assisting a human in picking tabletop objects for which the robot does not have prior CAD models. Experimental results demonstrate that in comparison with a baseline method that only focuses on stability, our method is able to provide suitable grasps for different tasks.

show abstract