Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Park, Kiru; Patten, Timothy; Vincze, Markus

doi:10.1109/iccv.2019.00776

Cited by 426 publications

(332 citation statements)

References 30 publications

Supporting

Mentioning

332

Contrasting

Order By: Relevance

“…Traditional methods identify hand-crafted features to localize an object model within a scene (Klank et al, 2009;Srinivasa et al, 2010;Chitta et al, 2012a) but more recently advances for pose estimation have been made by the application of deep learning (Xiang et al, 2018;Park et al, 2019b;Zakharov et al, 2019) and grasping pipelines achieve high success rate (Tremblay et al, 2018;Wang C. et al, 2019). The main limitation of this direction of research, however, is the closedworld assumption.…”

Section: Related Workmentioning

confidence: 99%

“…For the reconstruction of VD-NOC values, the standard L1 loss is applied for each pixel p. Since background pixels are masked out, their values are easy to predict. Hence, the loss values for pixels on the object masks M i ∈ R W×H are weighted by a factor of 3 to more precisely predict the values of pixels in the object masks (Park et al, 2019b). The reconstruction loss is thus given by,…”

Section: Training Objectivementioning

confidence: 99%

See 1 more Smart Citation

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

2020

Self Cite

View full text Add to dashboard Cite

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To transfer the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbor search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalized object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to transfer grasps to new instances as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritize grasp configurations that enable the functional use of objects.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Training Objectivementioning

confidence: 99%

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Euclidean distance, scalable nearest neighbor search method, and CNN are integrated as an efficient model to capture both the object identity and 3D pose [17,18]. Park et al proposed a novel architecture Pix2Pose based on CNN [19], which predicted the coordinates of each pixel after feature extraction, and then calculated the position and orientation by voting. Although this effort largely improved the robustness of pose estimation, especially under heavy occlusion, the computation cost was relatively expensive considering a comparable accuracy can be achieved with other methods.…”

Section: Related Workmentioning

confidence: 99%

Cognitive Template-Clustering Improved LineMod for Efficient Multi-object Pose Estimation

et al. 2020

View full text Add to dashboard Cite

Various types of theoretical algorithms have been proposed for 6D pose estimation, e.g., the point pair method, template matching method, Hough forest method, and deep learning method. However, they are still far from the performance of our natural biological systems, which can undertake 6D pose estimation of multi-objects efficiently, especially with severe occlusion. With the inspiration of the Müller-Lyer illusion in the biological visual system, in this paper, we propose a cognitive template-clustering improved LineMod (CT-LineMod) model. The model uses a 7D cognitive feature vector to replace standard 3D spatial points in the clustering procedure of Patch-LineMod, in which the cognitive distance of different 3D spatial points will be further influenced by the additional 4D information related with direction and magnitude of features in the Müller-Lyer illusion. The 7D vector will be dimensionally reduced into the 3D vector by the gradient-descent method, and then further clustered by K-means to aggregately match templates and automatically eliminate superfluous clusters, which makes the template matching possible on both holistic and part-based scales. The model has been verified on the standard Doumanoglou dataset and demonstrates a state-of-the-art performance, which shows the accuracy and efficiency of the proposed model on cognitive feature distance measurement and template selection on multiple pose estimation under severe occlusion. The powerful feature representation in the biological visual system also includes characteristics of the Müller-Lyer illusion, which, to some extent, will provide guidance towards a biologically plausible algorithm for efficient 6D pose estimation under severe occlusion.

show abstract

“…3. In this novel method Pix2Pose [16] we predict these coloured images to build a 2D-3D correspondence per pixel directly without any feature matching operation.…”

Section: Object Detection and Pose Estimationmentioning

confidence: 99%

“…In these cases the object pose will not be correct or wrongly attached to a symmetric view resulting in divergence of network training. The ideas presented in Pix2Pose tackle these problems [16]. For occlusion, the pixel-wise prediction is performed for not only the visible area but also the occluded region.…”

Section: Object Detection and Pose Estimationmentioning

confidence: 99%

Learn, detect, and grasp objects in real-world settings

Vincze

Patten

Park

et al. 2020

Elektrotech. Inftech.

Self Cite

View full text Add to dashboard Cite

Experts predict that future robot applications will require safe and predictable operation: robots will need to be able to explain what they are doing to be trusted. To reach this goal, they will need to perceive their environment and its object to better understand the world and the tasks they have to perform. This article gives an overview of present advances with the focus on options to learn, detect, and grasp objects. With the approach of colour and depth (RGB-D) cameras and the advances in AI and deep learning methods, robot vision has been pushed considerably over the last years. We summarise recent results for pose estimation of objects and work on verifying object poses using a digital twin and physics simulation. The idea is that any hypothesis from an object detector and pose estimator is verified leveraging on the continuous advances in deep learning approaches to create object hypotheses. We then show that the object poses are robust enough such that a mobile manipulator can approach the object and grasp it. We intend to indicate that it is now feasible to model, recognise and grasp many objects with good performance, though further work is needed for applications in industrial settings.

show abstract

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Cited by 426 publications

References 30 publications

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping

Cognitive Template-Clustering Improved LineMod for Efficient Multi-object Pose Estimation

Learn, detect, and grasp objects in real-world settings

Contact Info

Product

Resources

About