Learning from Demonstrations for Autonomous Soft-tissue Retraction

Abstract

The current research focus in Robot-Assisted Minimally Invasive Surgery (RAMIS) is directed towards increasing the level of robot autonomy, to place surgeons in a supervisory position. Although Learning from Demonstrations (LfD) approaches are among the preferred ways for an autonomous surgical system to learn expert gestures, they require a high number of demonstrations and show poor generalization to the variable conditions of the surgical environment. In this work, we propose an LfD methodology based on Generative Adversarial Imitation Learning (GAIL) that is built on a Deep Reinforcement Learning (DRL) setting. GAIL combines generative adversarial networks to learn the distribution of expert trajectories with a DRL setting to ensure generalisation of trajectories providing human-like behaviour. We consider automation of tissue retraction, a common RAMIS task that involves soft tissues manipulation to expose a region of interest. In our proposed methodology, a small set of expert trajectories can be acquired through the da Vinci Research Kit (dVRK) and used to train the proposed LfD method inside a simulated environment. Results indicate that our methodology can accomplish the tissue retraction task with human-like behaviour while being more sample-efficient than the baseline DRL method. Towards the end, we show that the learnt policies can be successfully transferred to the real robotic platform and deployed for soft tissue retraction on a synthetic phantom.

LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

“…Instead of exhaustively tuning the algorithm, hyperparameters, network architecture, and reward functions, the goal of these experiments is to show the effect of various environment configurations on task complexity, while keeping the learning setup constant. We train using Proximal Policy Optimization (PPO) (Schulman et al, 2017), as it is a popular algorithm that has been applied successfully to diverse problems in the literature (Andrychowicz et al, 2020;Mirhoseini et al, 2021;Pore et al, 2021b), thus making it suitable for use as a baseline. All environments were tested in several configurations, using either state or image observations, and varying up to two other environment-specific parameters.…”

“…In addition, images from the endoscopic camera are the primary source of information in surgical settings, but many existing environment suites do not support image observations. Recent works for automation in RALS propose algorithmic advances for solving surgical subtasks, but evaluate these algorithms in a) handcrafted simulation environments that are often not publicly available (Shin et al, 2019;Scheikl et al, 2021;He et al, 2022;Bourdillon et al, 2022) or b) on expensive robotic hardware such as the daVinci Research Kit (dVRK) (Tagliabue et al, 2020;Pore et al, 2021b;Chiu et al, 2021;Varier et al, 2020). This results in a high barrier for researchers to contribute to the field, and ultimately impedes vital algorithmic development.…”

Deep learning for semantic segmentation of organs and tissues in laparoscopic surgery

“…Tissue manipulation is a common surgical task that encompasses both direct and indirect manipulation of deformable objects. Examples for direct manipulation are tissue retraction to visualize occluded structures [4]- [6] or organ manipulation to bring a deformable object into a desired shape [7], [8]. Indirect manipulation involves moving specific landmarks on tissue by altering the tissue's shape through manipulation of other points on the tissue.…”

Sim-to-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery

“…Training in simulation enables end-to-end Reinforcement Learning (RL) of complex tasks in a safe and controlled environment, without requiring direct access to the real surgical robotic system. Training on real surgical robotic systems is impractical as RL algorithms often require millions of environment interactions to train a policy and unsafe behavior during training may damage the system [3].…”