Open Simulation Environment for Learning and Practice of Robot-Assisted Surgical Suturing

Munawar, Adnan; Wu, Jie; Fischer, Gregory S.; Taylor, Russell H.; Kazanzides, Peter

doi:10.1109/lra.2022.3146900

Cited by 16 publications

(7 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We distinguish between these two facets of simulation, the visual and the physical, since many simulation frameworks at this moment focus on one or the other. [98] Python, ROS ✓ Physics-based endoscopic simulator AMBF Drilling Simulator [28] C++, Python, ROS ✓ Physics-based microscopic simulator AMBF-RL [99] Python, ROS ✓ Physics-based endoscopic simulator VisionBlender [97] Blender [80] plug-in ✓ Physics-based monoscopic/stereo simulator SurRol [29] Python ✓ Physics-based endoscopic simulator…”

Section: Simulation Methods For Mismentioning

confidence: 99%

See 1 more Smart Citation

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

To mitigate the challenges of operating through narrow incisions under image guidance, there is a desire to develop intelligent systems that assist decision making and spatial reasoning in minimally invasive surgery (MIS). In this context, machine learning-based systems for interventional image analysis are receiving considerable attention because of their flexibility and the opportunity to provide immediate, informative feedback to clinicians. It is further believed that learning-based image analysis may eventually form the foundation for semior fully automated delivery of surgical treatments. A significant bottleneck in developing such systems is the availability of annotated images with sufficient variability to train generalizable models, particularly the most recently favored deep convolutional neural networks or transformer architectures. A popular alternative to acquiring and manually annotating data from the clinical practice is the simulation of these data from human-based models. Simulation has many advantages, including the avoidance of ethical issues, precisely controlled environments, and the scalability of data collection. Here, we survey recent work that relies on in silico training of learning-based MIS systems, in which data are generated via computational simulation. For each imaging modality, we review available simulation tools in terms of compute requirements, image quality, and usability, as well as their applications for training intelligent systems. We further discuss open challenges for simulation-based development of MIS systems, such as the need for integrated imaging and physical modeling for non-optical modalities, as well as generative patient models not dependent on underlying CT, MRI, or other patient data. In conclusion, as the capabilities of in silico training mature, with respect to sim-to-real transfer, computational efficiency, and degree of control, they are contributing toward the next generation of intelligent surgical systems.

show abstract

Section: Simulation Methods For Mismentioning

confidence: 99%

“…Surgeons and intelligent systems must also contend with smoke from energized devices [25,38] and blood. Finally, in order to facilitate machine learning, it is desirable for simulation frameworks to provide various ground truth data by default, including segmentation maps, depth maps, object poses, camera pose, and more [98].…”

Section: Visible Light Imagingmentioning

confidence: 99%

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…Drilling and cutting of the bone 17 have been based on the open source iMSTK framework. Recently, a standardized simulation platform for training and testing algorithms to control surgical robotic systems, and suturing in particular, has been announced 18 …”

Section: Soft Tissue Simulation and Vr Trainersmentioning

confidence: 99%

“…Recently, a standardized simulation platform for training and testing algorithms to control surgical robotic systems, and suturing in particular, has been announced. 18 Several commercial VR training environments aim to reduce time spent teaching in the OR by offering training modules with virtual anatomy that can be probed using force feedback devices. However, commercial training environments neither aim to capture the broad spectrum of physical variations encountered in laparoscopic practice nor prepare learners for less common interventions.…”

Section: Soft Tissue Simulation and Vr Trainersmentioning

confidence: 99%

Adding Safety Rules to Surgeon-Authored Virtual Reality Training

et al. 2023

View full text Add to dashboard Cite

Introduction:The Toolkit for Illustration of Procedures in Surgery (TIPS) is an open source virtual reality (VR) laparoscopic simulation-based training environment with force feedback. The TIPS-author is a content creation interface that allows a surgeon educator (SE) to assemble new laparoscopic training modules. New technology enables safety rules to be specified by the SE, automatically tracks specified safety errors, and summarizes and communicates achievements and errors to the surgical trainee. Methods: The TIPS-author combines and initializes building blocks of anatomy with their physical properties, as selected by the SE from a database. The SE can add any safety rule that can be tested in terms of location, proximity, separation, clip count, and force. Errors are then automatically monitored during simulation and recorded as visual snapshots for feedback to the trainee. The TIPS was field tested at 2 surgical conferences, one before and one after adding the error snapshot feature. Results: Sixty-four respondents at 2 surgical conferences assessed the utility of TIPS on a Likert scale. While other ratings remained unchanged for an overall score of 5.24 of 7 (7 = very useful), the rating of the statement "The TIPS interface helps learners understand the force necessary to explore the anatomy" improved from 5.04 to 5.35 of 7 after the snapshot mechanism was added. Conclusions:The ratings indicate the viability of the TIPS open source SE-authored surgical training units with safety rules. Presenting SE-determined procedural missteps via the snapshot mechanism at the end of the training increases perceived utility.

show abstract

“…Moreover, AMBF contains a built-in peg transfer board that could already serve as a benchmark. Important to note the 2021 AccelNet Surgical Robotics Challenge , also based on the AMBF simulator, offers a simulated setup to develop autonomous suturing applications [ 89 ].…”

Section: Benchmarking Techniquesmentioning

confidence: 99%

Performance and Capability Assessment in Surgical Subtask Automation

Nagy

Haidegger

2022

Sensors

View full text Add to dashboard Cite

Robot-Assisted Minimally Invasive Surgery (RAMIS) has reshaped the standard clinical practice during the past two decades. Many believe that the next big step in the advancement of RAMIS will be partial autonomy, which may reduce the fatigue and the cognitive load on the surgeon by performing the monotonous, time-consuming subtasks of the surgical procedure autonomously. Although serious research efforts are paid to this area worldwide, standard evaluation methods, metrics, or benchmarking techniques are still not formed. This article aims to fill the void in the research domain of surgical subtask automation by proposing standard methodologies for performance evaluation. For that purpose, a novel characterization model is presented for surgical automation. The current metrics for performance evaluation and comparison are overviewed and analyzed, and a workflow model is presented that can help researchers to identify and apply their choice of metrics. Existing systems and setups that serve or could serve as benchmarks are also introduced and the need for standard benchmarks in the field is articulated. Finally, the matter of Human–Machine Interface (HMI) quality, robustness, and the related legal and ethical issues are presented.

show abstract

Open Simulation Environment for Learning and Practice of Robot-Assisted Surgical Suturing

Cited by 16 publications

References 34 publications

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

In silico simulation: a key enabling technology for next-generation intelligent surgical systems

Adding Safety Rules to Surgeon-Authored Virtual Reality Training

Performance and Capability Assessment in Surgical Subtask Automation

Contact Info

Product

Resources

About