Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry

Dong, Siyuan; Jha, Devesh K.; Romeres, Diego; Kim, Sangwoon; Nikovski, Daniel; Rodríguez, Alberto

doi:10.1109/icra48506.2021.9561646

Cited by 76 publications

(42 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…RL endows robots the promise to accommodate variations in environmental configurations. Dong et al [20] developed a tactile-based RL algorithm for insertion tasks. Oikawa used a non-diagonal stiffness matrix for precise assembly [21].…”

Section: B Rl-based Manipulationmentioning

confidence: 99%

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Jin¹,

Lin²,

Tan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Combined visual and force feedback play an essential role in contact-rich robotic manipulation tasks. Current methods focus on developing the feedback control around a single modality while underrating the synergy of the sensors. Fusing different sensor modalities is necessary but remains challenging. A key challenge is to achieve an effective multimodal and generalized control scheme to novel objects with precision. This paper proposes a practical multi-modal sensor fusion mechanism using hierarchical policy learning. To begin with, we use a self-supervised encoder that extracts multi-view visual features and a hybrid motion/force controller that regulates force behaviors. Next, the multi-modality fusion is simplified by hierarchical integration of the vision, force, and proprioceptive data in the reinforcement learning (RL) algorithm. Moreover, with hierarchical policy learning, the control scheme can exploit the visual feedback limits and explore the contribution of individual modality in precise tasks. Experiments indicate that robots with the control scheme could assemble objects with 0.25mm clearance in simulation. The system could be generalized to widely varied initial configurations and new shapes. Experiments validate that the simulated system can be robustly transferred to reality without fine-tuning.

show abstract

Section: B Rl-based Manipulationmentioning

confidence: 99%

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Jin¹,

Lin²,

Tan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In this area, most prior research either focuses on contact configuration control assuming a known model of the world [2], [8], [9], or contact configuration estimation assuming stable interactions [10]. Work on joint estimation and control either uses simplified (e.g., frictionless) models of contact [11], [12], or learns task-specific policies from data (e.g, for cable manipulation [3] or part insertion [13]). Our contribution is an object-agnostic joint estimation and control framework that reasons about all frictional interactions between the robot, object, and environment.…”

Section: Related Workmentioning

confidence: 99%

Manipulation of unknown objects via contact configuration regulation

Doshi¹,

Taylor²,

Rodríguez³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present an approach to robotic manipulation of unknown objects through regulation of the object's contact configuration: the location, geometry, and mode of all contacts between the object, robot, and environment. A contact configuration constrains the forces and motions that can be applied to the object; however, synthesizing these constraints generally requires knowledge of the object's pose and geometry. We develop an object-agnostic approach for estimation and control that circumvents this need. Our framework directly estimates a set of wrench and motion constraints which it uses to regulate the contact configuration. We use this to reactively manipulate unknown planar objects in the gravity plane. A video describing our work can be found on our project page: http://mcube. mit.edu/research/contactConfig.html.

show abstract

“…They consider a decomposition of the control task in object state control and contact state control. The contact state was detected using vision-based tactile sensors [19], [20], [21]. As the task mostly required sticking contact for stability, the tactile feedback was designed to make corrections to push the system away from the boundary of friction cone at the different contact locations.…”

Section: Related Workmentioning

confidence: 99%

Robust Pivoting: Exploiting Frictional Stability Using Bilevel Optimization

Shirai¹,

Jha²,

Raghunathan³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Generalizable manipulation requires that robots be able to interact with novel objects and environment. This requirement makes manipulation extremely challenging as a robot has to reason about complex frictional interaction with uncertainty in physical properties of the object. In this paper, we study robust optimization for control of pivoting manipulation in the presence of uncertainties. We present insights about how friction can be exploited to compensate for the inaccuracies in the estimates of the physical properties during manipulation. In particular, we derive analytical expressions for stability margin provided by friction during pivoting manipulation. This margin is then used in a bilevel trajectory optimization algorithm to design a controller that maximizes this stability margin to provide robustness against uncertainty in physical properties of the object. We demonstrate our proposed method using a 6 DoF manipulator for manipulating several different objects.

show abstract

Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry

Cited by 76 publications

References 14 publications

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning

Manipulation of unknown objects via contact configuration regulation

Robust Pivoting: Exploiting Frictional Stability Using Bilevel Optimization

Contact Info

Product

Resources

About