Rigid 3D Registration of Pre-operative Information for Semi-Autonomous Surgery

Piccinelli, Nicola; Roberti, Andrea; Tagliabue, Eleonora; Setti, Francesco; Kronreif, Gernot; Muradore, Riccardo; Fiorini, Paolo

doi:10.1109/ismr48331.2020.9312949

Cited by 6 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before proceeding with the execution of the surgical procedure, all the robotic arms and the vision system are registered together to create a common reference frame following the methodology proposed in [33]. Moreover, the operative scene is reconstructed in 3D to identify pre-operative points of interest that are mapped to a live point cloud reconstruction during the operation [34]. The latter allows to precisely track all exogenous actors, for instance the catheter.…”

Section: Experimental Validationmentioning

confidence: 99%

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Falezza

Piccinelli

Rossi

et al. 2021

IEEE Trans. Med. Robot. Bionics

Self Cite

View full text Add to dashboard Cite

In this paper we propose a new methodology to model surgical procedures that is specifically tailored to semiautonomous robotic surgery. We propose to use a restricted version of statecharts to merge the bottom-up approach, based on data-driven techniques (e.g. machine learning), with the top-down approach based on knowledge representation techniques. We consider medical knowledge about the procedure and sensing of the environment in two concurrent regions of the statecharts to facilitate re-usability and adaptability of the modules. Our approach allows producing a well defined procedural model exploiting the hierarchy capability of the statecharts, while machine learning modules act as soft sensors to trigger state transitions. Integrating data driven and prior knowledge techniques provides a robust, modular, flexible and re-configurable methodology to define a surgical procedure which is comprehensible by both humans and machines. We validate our approach on the three surgical phases of a Robot-Assisted Radical Prostatectomy (RARP) that directly involve the assistant surgeon: bladder mobilization, bladder neck transection, and vesicourethral anastomosis, all performed on synthetic manikins.

show abstract

Section: Experimental Validationmentioning

confidence: 99%

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Falezza

Piccinelli

Rossi

et al. 2021

IEEE Trans. Med. Robot. Bionics

Self Cite

View full text Add to dashboard Cite

show abstract

“…3 bottom) is precisely aligned with the simulated scene (Fig. 3 top) by mapping the poses of the PSM in the two environments, as described in [20]. Accurate registration is essential to prevent inconsistencies between the two environments, especially considering that all the movements of the dVRK arm in the real system are controlled through the simulated robot, including the grasping action.…”

Section: B Real Robotic Experimentsmentioning

confidence: 99%

Learning from Demonstrations for Autonomous Soft-tissue Retraction

Pore¹,

Tagliabue²,

Piccinelli³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The current research focus in Robot-Assisted Minimally Invasive Surgery (RAMIS) is directed towards increasing the level of robot autonomy, to place surgeons in a supervisory position. Although Learning from Demonstrations (LfD) approaches are among the preferred ways for an autonomous surgical system to learn expert gestures, they require a high number of demonstrations and show poor generalization to the variable conditions of the surgical environment. In this work, we propose an LfD methodology based on Generative Adversarial Imitation Learning (GAIL) that is built on a Deep Reinforcement Learning (DRL) setting. GAIL combines generative adversarial networks to learn the distribution of expert trajectories with a DRL setting to ensure generalisation of trajectories providing human-like behaviour. We consider automation of tissue retraction, a common RAMIS task that involves soft tissues manipulation to expose a region of interest. In our proposed methodology, a small set of expert trajectories can be acquired through the da Vinci Research Kit (dVRK) and used to train the proposed LfD method inside a simulated environment. Results indicate that our methodology can accomplish the tissue retraction task with human-like behaviour while being more sample-efficient than the baseline DRL method. Towards the end, we show that the learnt policies can be successfully transferred to the real robotic platform and deployed for soft tissue retraction on a synthetic phantom.

show abstract

“…Specifically, advances in medical imaging and vision techniques have significantly improved the performance of robotic surgical systems in a range of clinical scenarios, such as orthopaedics and neurosurgery [4]. Vision systems can retrieve pre and intra operative information from tomography (CT) [5], magnetic resonance (MR) and ultrasound to guide trajectory execution and support the surgeon in decision making. However, in order to perform image-guided interventions, an accurate calibration is necessary to represent poses of robots, instruments and anatomy in a common reference frame.…”

Section: Introductionmentioning

confidence: 99%

Improving Rigid 3-D Calibration for Robotic Surgery

Roberti

Piccinelli

Meli

et al. 2020

IEEE Trans. Med. Robot. Bionics

Self Cite

View full text Add to dashboard Cite

Autonomy is the next frontier of research in robotic surgery and its aim is to improve the quality of surgical procedures in the next future. One fundamental requirement for autonomy is advanced perception capability through vision sensors. In this paper, we propose a novel calibration technique for a surgical scenario with a da Vinci R Research Kit (dVRK) robot. Camera and robotic arms calibration are necessary to precise position and emulate expert surgeon. The novel calibration technique is tailored for RGB-D cameras. Different tests performed on relevant use cases prove that we significantly improve precision and accuracy with respect to state of the art solutions for similar devices on a surgical-size setups. Moreover, our calibration method can be easily extended to standard surgical endoscope used in real surgical scenario.

show abstract

Rigid 3D Registration of Pre-operative Information for Semi-Autonomous Surgery

Cited by 6 publications

References 24 publications

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Modeling of Surgical Procedures Using Statecharts for Semi-Autonomous Robotic Surgery

Learning from Demonstrations for Autonomous Soft-tissue Retraction

Improving Rigid 3-D Calibration for Robotic Surgery

Contact Info

Product

Resources

About