I Like to Move It: 6D Pose Estimation as an Action Decision Process

Busam, Benjamin; Jung, Hyun Jun; Navab, Nassir

doi:10.48550/arxiv.2009.12678

Cited by 8 publications

(10 citation statements)

References 72 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The superior performance of SE(3)-TrackerNet mostly comes from the rotation-translation disentanglement scheme, a Lie Algebra rotation representation and the domain randomization data enhancement strategy [69]. Beyond the above methods, Busam et al [151] propose "I Like to Move It", which learns a decision process to iteratively determine an acceptable final pose. They build a decision process where an initial pose is updated in incremental discrete steps.…”

Section: Instance Level Monocular Object Pose Trackingmentioning

confidence: 99%

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

Fan¹,

Zhu²,

He³

et al. 2021

Preprint

View full text Add to dashboard Cite

Object pose detection and tracking has recently attracted increasing attention due to its wide applications in many areas, such as autonomous driving, robotics, and augmented reality. Among methods for object pose detection and tracking, deep learning is the most promising one that has shown better performance than others. However, there is lack of survey study about latest development of deep learning based methods. Therefore, this paper presents a comprehensive review of recent progress in object pose detection and tracking that belongs to the deep learning technical route. To achieve a more thorough introduction, the scope of this paper is limited to methods taking monocular RGB/RGBD data as input, covering three kinds of major tasks: instance-level monocular object pose detection, category-level monocular object pose detection, and monocular object pose tracking. In our work, metrics, datasets, and methods about both detection and tracking are presented in detail. Comparative results of current state-of-the-art methods on several publicly available datasets are also presented, together with insightful observations and inspiring future research directions.

show abstract

Section: Instance Level Monocular Object Pose Trackingmentioning

confidence: 99%

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

Fan¹,

Zhu²,

He³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…While the accuracy of estimating the 6D pose keeps steadily increasing, it also comes with a heavy burden in annotating data for 3D CAD model and 6D pose, which scales poorly to multiple objects, often rendering it impractical for real applications [37], [38], [39]. In contrast, our method only needs a short live demonstration of an human-object interaction in order to reliably interact with novel objects.…”

Section: A Model-based Graspingmentioning

confidence: 99%

DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration

Wang¹,

Manhardt²,

Minciullo³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The ability to successfully grasp objects is crucial in robotics, as it enables several interactive downstream applications. To this end, most approaches either compute the full 6D pose for the object of interest or learn to predict a set of grasping points. While the former approaches do not scale well to multiple object instances or classes yet, the latter require large annotated datasets and are hampered by their poor generalization capabilities to new geometries. To overcome these shortcomings, we propose to teach a robot how to grasp an object with a simple and short human demonstration. Hence, our approach neither requires many annotated images nor is it restricted to a specific geometry. We first present a small sequence of RGB-D images displaying a human-object interaction. This sequence is then leveraged to build associated hand and object meshes that represent the depicted interaction. Subsequently, we complete missing parts of the reconstructed object shape and estimate the relative transformation between the reconstruction and the visible object in the scene. Finally, we transfer the a-priori knowledge from the relative pose between object and human hand with the estimate of the current object pose in the scene into necessary grasping instructions for the robot. Exhaustive evaluations with Toyota's Human Support Robot (HSR) in real and synthetic environments demonstrate the applicability of our proposed methodology and its advantage in comparison to previous approaches.

show abstract

“…Reinforcement Learning in Object Pose Estimation: In the related domain of object pose estimation, Krull et al [11] use RL to find a policy that efficiently allocates refinement iterations to a pool of pose hypotheses. Closely related to our approach, in RGB-based object pose estimation [24,5], RL is used to train policies that manipulate an object's pose. Based on 2D segmentation masks, these agents learn to predict discrete refinement actions.…”

Section: Related Workmentioning

confidence: 99%

“…In an effort to bridge this gap between methods, we design a novel registration approach that unifies accuracy, robustness to noise and initialization with inference speed. While reinforcement learning methods for RGB-based object pose refinement are proposed [24,5], to the best of our knowledge, we are the first to consider 3D point cloud registration as a reinforcement learning problem. Our approach is based on a combination of Imitation Learning (IL) and Reinforcement Learning (RL); imitating an expert to learn an accurate initial policy, reinforcing a symmetry-invariant reward to further improve the policy.…”

Section: Introductionmentioning

confidence: 99%

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

Bauer¹,

Patten²,

Vincze³

2021

Preprint

View full text Add to dashboard Cite

Point cloud registration is a common step in many 3D computer vision tasks such as object pose estimation, where a 3D model is aligned to an observation. Classical registration methods generalize well to novel domains but fail when given a noisy observation or a bad initialization. Learning-based methods, in contrast, are more robust but lack in generalization capacity. We propose to consider iterative point cloud registration as a reinforcement learning task and, to this end, present a novel registration agent (ReAgent). We employ imitation learning to initialize its discrete registration policy based on a steady expert policy. Integration with policy optimization, based on our proposed alignment reward, further improves the agent's registration performance. We compare our approach to classical and learning-based registration methods on both ModelNet40 (synthetic) and ScanObjectNN (real data) and show that our ReAgent achieves state-of-the-art accuracy. The lightweight architecture of the agent, moreover, enables reduced inference time as compared to related approaches. In addition, we apply our method to the object pose estimation task on real data (LINEMOD), outperforming state-of-the-art pose refinement approaches. Code is available at github.com/dornik/reagent.

show abstract

I Like to Move It: 6D Pose Estimation as an Action Decision Process

Cited by 8 publications

References 72 publications

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

Deep Learning on Monocular Object Pose Detection and Tracking: A Comprehensive Overview

DemoGrasp: Few-Shot Learning for Robotic Grasping with Human Demonstration

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

Contact Info

Product

Resources

About