Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor

Seita, Daniel; Ganapathi, Aditya; Hoque, Ryan; Hwang, Mina; Cen, Edward; Tanwani, Ajay Kumar; Balakrishna, Ashwin; Ichnowski, Jeffrey; Jamali, Nawid; Yamane, Katsu; Iba, Soshi; Canny, John; Goldberg, Ken

doi:10.1109/iros45743.2020.9341608

Cited by 87 publications

(92 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This application category includes papers working on cutting and debridement [61], [87], [90], [92], [95], [97] as well as the retraction and dissection of tissues [101], [131], [132], [155], [257], [265] or blood suction [226]. Also included is tissue palpation for locating tumors or vessels and more general tissue manipulation, as in [13]- [15], [17], [82], [83], [86], [98], [99], [154], [164], [225], and [241], with experiments sometimes using just common fabric as a phantom for tissue [94], [100].…”

Section: Instrument Controlmentioning

confidence: 99%

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

D'Ettorre

Mariani

Stilli

et al. 2021

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.R obotic-assisted surgery is now well established in clinical practice and has become the gold-standard clinical treatment option for several clinical indications. The field of robotic-assisted surgery is expected to grow substantially in the next decade, with a range of new robotic devices emerging to address unmet clinical needs across different specialties. A vibrant surgical robotics research community is pivotal for conceptualizing such new systems as well as for developing and training the engineers and scientists to translate them into practice. The da Vinci Research Kit (dVRK), an academic and industry collaborative effort to repurpose decommissioned da Vinci surgical systems [Intuitive Surgical Inc. (ISI), California, USA] as a research platform for surgical robotics research, has been a key initiative for addressing a barrier to entry for new research groups in surgical robotics. In this article, we present an extensive review of the publications that have been facilitated by the dVRK over the past decade. We classify research efforts into different categories and outline some of the major challenges and needs for the robotics community to maintain and build upon this initiative.

show abstract

Section: Instrument Controlmentioning

confidence: 99%

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

D'Ettorre

Mariani

Stilli

et al. 2021

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

show abstract

“…Deformable object manipulation has seen a surge of interest recently in the robotics research community [18,34,33,28,6,10,25,26], but modeling state and action spaces in such tasks remains challenging. Much prior work focuses on the complex task of generating plans for manipulating cables and higher-dimensional deformable objects.…”

Section: Related Workmentioning

confidence: 99%

“…Other methods learn visuomotor policies end-to-end. Imitation learning [25], video prediction models [10], model-free reinforcement learning (RL) [17], and model-based RL [20] have successfully been applied to goal-conditioned fabric manipulation. In other end-to-end approaches, Nair et al [21] learn a dynamics model for performing template shape matching and knot-tying with rope.…”

Section: A Deformable Object Manipulationmentioning

confidence: 99%

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies

Sundaresan

Grannen

Thananjeyan

et al. 2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

Robot manipulation for untangling 1D deformable structures such as ropes, cables, and wires is challenging due to their infinite dimensional configuration space, complex dynamics, and tendency to self-occlude. Analytical controllers often fail in the presence of dense configurations, due to the difficulty of grasping between adjacent cable segments. We present two algorithms that enhance robust cable untangling, LOKI and SPi-DERMan, which operate alongside HULK, a high-level planner from prior work. LOKI uses a learned model of manipulation features to refine a coarse grasp keypoint prediction to a precise, optimized location and orientation, while SPiDERMan uses a learned model to sense task progress and apply recovery actions. We evaluate these algorithms in physical cable untangling experiments with 336 knots and over 1500 actions on real cables using the da Vinci surgical robot. We find that the combination of HULK, LOKI, and SPiDERMan is able to untangle dense overhand, figure-eight, double-overhand, square, bowline, granny, stevedore, and triple-overhand knots. The composition of these methods successfully untangles a cable from a dense initial configuration in 68.3% of 60 physical experiments and achieves 50% higher success rates than baselines from prior work. Supplementary material, code, and videos can be found at https://tinyurl.com/rssuntangling.

show abstract

“…Hence, dealing with deformable objects planning is a significant issue. In this issue, there have been many studies that seek to handle deformable object planning problems [ 3 , 4 , 5 ].…”

Section: Introductionmentioning

confidence: 99%

“…Another category of studies, i.e., model-based learning, has also shown promising in sample-efficient learning [ 13 , 14 ]. Using such model-based learning studies for deformable objects, however, researchers should consider how to represent state and learn action models appropriately.…”

Section: Introductionmentioning

confidence: 99%

Action Generative Networks Planning for Deformable Object with Raw Observations

Sheng

Jin

et al. 2021

Sensors

View full text Add to dashboard Cite

Synthesizing plans for a deformable object to transit from initial observations to goal observations, both of which are represented by high-dimensional data (namely “raw” data), is challenging due to the difficulty of learning abstract state representations of raw data and transition models of continuous states and continuous actions. Even though there have been some approaches making remarkable progress regarding the planning problem, they often neglect actions between observations and are unable to generate action sequences from initial observations to goal observations. In this paper, we propose a novel algorithm framework, namely AGN. We first learn a state-abstractor model to abstract states from raw observations, a state-generator model to generate raw observations from states, a heuristic model to predict actions to be executed in current states, and a transition model to transform current states to next states after executing specific actions. After that, we directly generate plans for a deformable object by performing the four models. We evaluate our approach in continuous domains and show that our approach is effective with comparison to state-of-the-art algorithms.

show abstract

Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisor

Cited by 87 publications

References 39 publications

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies

Action Generative Networks Planning for Deformable Object with Raw Observations

Contact Info

Product

Resources

About