Active articulation model estimation through interactive perception

Hausman, Karol; Niekum, Scott; Osentoski, Sarah; Sukhatme, Gaurav S.

doi:10.1109/icra.2015.7139655

Cited by 78 publications

(53 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It consists of a policy network and a Q-value network, both implemented as a Multi-layer Perception (MLP). The policy network receives the state as input (R 33 ), and predicts the residual gripper pose as action (R 6 ). Its network architecture is implemented as a 4-layer MLP (33 → 512 → 512 → 512 → 512), followed by two separate fully connected layers (512 → 6) that estimate the mean and variance of action probabilities respectively.…”

Section: Discussionmentioning

confidence: 99%

“…Perceiving and Manipulating 3D Articulated Objects has been a long-lasting research topic in computer vision and robotics. A vast literature [23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41] has demonstrated successful systems, powered by visual feature trackers, motion segmentation predictors, and probabilistic estimators, for obtaining accurate link poses, joint parameters, kinematic structures, and even system dynamics of 3D articulated objects.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

Wu¹,

Zhao²,

Mo³

et al. 2021

Preprint

View full text Add to dashboard Cite

Perceiving and manipulating 3D articulated objects (e.g., cabinets, doors) in human environments is an important yet challenging task for future home-assistant robots. The space of 3D articulated objects is exceptionally rich in their myriad semantic categories, diverse shape geometry, and complicated part functionality. Previous works mostly abstract kinematic structure with estimated joint parameters and part poses as the visual representations for manipulating 3D articulated objects. In this paper, we propose object-centric actionable visual priors as a novel perception-interaction handshaking point that the perception system outputs more actionable guidance than kinematic structure estimation, by predicting dense geometry-aware, interaction-aware, and task-aware visual action affordance and trajectory proposals. We design an interaction-for-perception framework VAT-MART to learn such actionable visual representations by simultaneously training a curiosity-driven reinforcement learning policy exploring diverse interaction trajectories and a perception module summarizing and generalizing the explored knowledge for pointwise predictions among diverse shapes. Experiments prove the effectiveness of the proposed approach using the large-scale PartNet-Mobility dataset in SAPIEN environment and show promising generalization capabilities to novel test shapes, unseen object categories, and real-world data. Please check the project webpage for code, data, video, and more materials.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

Wu¹,

Zhao²,

Mo³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Instead of solely rely on the perception, the constrained object model can be obtained or refined by robot interactive perception. Nekum [11] aims to extract a constrained object model in 3D space given the observation with occlusion which can be clarified by robot interaction. Other methods discover and learn the object model learning totally from scratch by interaction [12].…”

Section: Related Workmentioning

confidence: 99%

Learning Descriptor of Constrained Task from Demonstration

Zhang,

Saveriano,

Piater

2021

Preprint

View full text Add to dashboard Cite

Constrained objects, such as doors and drawers are often complex and share a similar structure in the human environment. A robot needs to interact accurately with constrained objects to safely and successfully complete a task. Learning from Demonstration offers an appropriate path to learn the object structure of the demonstration for unknown objects for unknown tasks. There is work that extracts the kinematic model from motion. However, the gap remains when the robot faces a new object with a similar model but different contexts, e.g. size, appearance, etc. In this paper, we propose a framework that integrates all the information needed to learn a constrained motion from a depth camera into a descriptor of the constrained task. The descriptor consists of object information, grasping point model, constrained model, and reference frame model. By associating constrained learning and reference frame with the constrained object, we demonstrate that the robot can learn the book opening model and parameter of the constraints from demonstration and generalize to novel books.

show abstract

“…Building on the previous work, a unified framework with several extensions like dealing with kinematic loops and an extended set of experiments is presented in [10]. A particle filter based approach presented in [11] integrates the idea of interactive perception into a probabilistic framework using visual observations and manipulation feedback from the robot. They also presented best action selection methods based on entropy and information gain which guides the robot to perform the most useful interactions with the object to reduce the uncertainty on articulation model estimates.…”

Section: Introductionmentioning

confidence: 99%

Momentum-Based Topology Estimation of Articulated Objects

Tirupachuri

Traversaro

Nori

et al. 2019

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

Articulated objects like doors, drawers, valves, and tools are pervasive in our everyday unstructured dynamic environments. Articulation models describe the joint nature between the different parts of an articulated object. As most of these objects are passive, a robot has to interact with them to infer all the articulation models to understand the object topology. We present a general algorithm to estimate the inherent articulation models by exploiting the momentum of the articulated system along with the interaction wrench while manipulating the object. We validate our approach with experiments in a simulation environment.

show abstract

Active articulation model estimation through interactive perception

Cited by 78 publications

References 20 publications

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects

Learning Descriptor of Constrained Task from Demonstration

Momentum-Based Topology Estimation of Articulated Objects

Contact Info

Product

Resources

About