Purpose: Single Incision Laparoscopic Surgery (SILS) decreases post-operative infections, but introduces limitations in the surgeon's manoeuverability and in the surgical field of view. This work aims at enhancing intraoperative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of pre-operative tissue models is updated online. A critical process involves the intraoperative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process.Methods: Two methods for soft tissue 3D reconstruction are proposed: Method 1 follows the traditional approach of the block matching algorithm. Method 2 performs a non-parametric Modified Census Transform to be more robust to illumination variation. The Simple Linear Iterative Clustering (SLIC) super pixel al- gorithm is exploited for disparity refinement by filling holes in the disparity images.Results: The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 mm and 1.66 mm respectively. A comparison with ground truth data demonstrated the disparity refinement procedure: (i) increases the number of reconstructed points by up to 43%; (ii) does not a↵ect the accuracy of the 3D reconstructions significantly.Conclusion: Both methods give results that compare favourably with the state-of-the-art methods. The computational time constraints their applicability in realtime, but can be greatly improved by using a GPU implementation.

The EndoAbS dataset contributes to an increase the number and variety of openly available datasets of surgical stereo images, including a highly accurate RF and different surgical conditions.

In abdominal surgery, intraoperative bleeding is one of the major complications that affect the outcome of minimally invasive surgical procedures. One of the causes is attributed to accidental damages to arteries or veins, and one of the possible risk factors falls on the surgeon's skills. This paper presents the development and application of an Enhanced Vision System for Robotic Surgery (EnViSoRS), based on a user-defined Safety Volume (SV) tracking to minimize the risk of intraoperative bleeding. It aims at enhancing the surgeon's capabilities by providing Augmented Reality (AR) assistance toward the protection of vessels from injury during the execution of surgical procedures with a robot. The core of the framework consists in (i) a hybrid tracking algorithm (LT-SAT tracker) that robustly follows a user-defined Safety Area (SA) in long term; (ii) a dense soft tissue 3D reconstruction algorithm, necessary for the computation of the SV; (iii) AR features for visualization of the SV to be protected and of a graphical gage indicating the current distance between the instruments and the reconstructed surface. EnViSoRS was integrated with a commercial robotic surgical system (the dVRK system) for testing and validation. The experiments aimed at demonstrating the accuracy, robustness, performance, and usability of EnViSoRS during the execution of a simulated surgical task on a liver phantom. Results show an overall accuracy in accordance with surgical requirements (<5 mm), and high robustness in the computation of the SV in terms of precision and recall of its identification. The optimization strategy implemented to speed up the computational time is also described and evaluated, providing AR features update rate up to 4 fps, without impacting the real-time visualization of the stereo endoscopic video. Finally, qualitative results regarding the system usability indicate that the proposed system integrates well with the commercial surgical robot and has indeed potential to offer useful assistance during real surgeries.

Several studies have demonstrated that training with a laparoscopic simulator improves laparoscopic technical skills. We describe how to build a homemade, low-cost laparoscopic training simulator (LABOT) and its validation as a training instrument. First, sixty surgeons filled out a survey characterized by 12 closed-answer questions about realism, ergonomics, and usefulness for surgical training (global scores ranged from 1—very insufficient to 5—very good). The results of the questionnaires showed a mean (±SD) rating score of 4.18 ± 0.65 for all users. Then, 15 students (group S) and 15 residents (group R) completed 3 different tasks (T1, T2, T3), which were repeated twice to evaluate the execution time and the number of users’ procedural errors. For T1, the R group had a lower mean execution time and a lower rate of procedural errors than the S group; for T2, the R and S groups had a similar mean execution time, but the R group had a lower rate of errors; and for T3, the R and S groups had a similar mean execution time and rate of errors. On a second attempt, all the participants tended to improve their results in doing these surgical tasks; nevertheless, after subgroup analysis of the T1 results, the S group had a better improvement of both parameters. Our laparoscopic simulator is simple to build, low-cost, easy to use, and seems to be a suitable resource for improving laparoscopic skills. In the future, further studies should evaluate the potential of this laparoscopic box on long-term surgical training with more complex tasks and simulation attempts.

Despite the benefits introduced by robotic systems in abdominal Minimally Invasive Surgery (MIS), major complications can still affect the outcome of the procedure, such as intra-operative bleeding. One of the causes is attributed to accidental damages to arteries or veins by the surgical tools, and some of the possible risk factors are related to the lack of sub-surface visibilty. Assistive tools guiding the surgical gestures to prevent these kind of injuries would represent a relevant step towards safer clinical procedures. However, it is still challenging to develop computer vision systems able to fulfill the main requirements: (i) long term robustness, (ii) adaptation to environment/object variation and (iii) real time processing. The purpose of this paper is to develop computer vision algorithms to robustly track soft tissue areas (Safety Area, SA), defined intra-operatively by the surgeon based on the real-time endoscopic images, or registered from a pre-operative surgical plan. We propose a framework to combine an optical flow algorithm with a tracking-by-detection approach in order to be robust against failures caused by: (i) partial occlusion, (ii) total occlusion, (iii) SA out of the field of view, (iv) deformation, (v) illumination changes, (vi) abrupt camera motion, (vii), blur and (viii) smoke. A Bayesian inference-based approach is used to detect the failure of the tracker, based on online context information. A Model Update Strategy (MUpS) is also proposed to improve the SA re-detection after failures, taking into account the changes of appearance of the SA model due to contact with instruments or image noise. The performance of the algorithm was assessed on two datasets, representing ex-vivo organs and in-vivo surgical scenarios. Results show that the proposed framework, enhanced with MUpS, is capable of maintain high tracking performance for extended periods of time ( ≃ 4 min - containing the aforementioned events) with high precision (0.7) and recall (0.8) values, and with a recovery time after a failure between 1 and 8 frames in the worst case.

Laser microsurgery is the current gold standard surgical technique for the treatment of selected diseases in delicate organs such as the larynx. However, the operations require large surgical expertise and dexterity, and face significant limitations imposed by available technology, such as the requirement for direct line of sight to the surgical field, restricted access, and direct manual control of the surgical instruments. To change this status quo, the European project μRALP pioneered research towards a complete redesign of current laser microsurgery systems, focusing on the development of robotic micro-technologies to enable endoscopic operations. This has fostered awareness and interest in this field, which presents a unique set of needs, requirements and constraints, leading to research and technological developments beyond μRALP and its research consortium. This paper reviews the achievements and key contributions of such research, providing an overview of the current state of the art in robot-assisted endoscopic laser microsurgery. The primary target application considered is phonomicrosurgery, which is a representative use case involving highly challenging microsurgical techniques for the treatment of glottic diseases. The paper starts by presenting the motivations and rationale for endoscopic laser microsurgery, which leads to the introduction of robotics as an enabling technology for improved surgical field accessibility, visualization and management. Then, research goals, achievements, and current state of different technologies that can build-up to an effective robotic system for endoscopic laser microsurgery are presented. This includes research in micro-robotic laser steering, flexible robotic endoscopes, augmented imaging, assistive surgeon-robot interfaces, and cognitive surgical systems. Innovations in each of these areas are shown to provide sizable progress towards more precise, safer and higher quality endoscopic laser microsurgeries. Yet, major impact is really expected from the full integration of such individual contributions into a complete clinical surgical robotic system, as illustrated in the end of this paper with a description of preliminary cadaver trials conducted with the integrated μRALP system. Overall, the contribution of this paper lays in outlining the current state of the art and open challenges in the area of robot-assisted endoscopic laser microsurgery, which has important clinical applications even beyond laryngology.

Abstract-In narrow band (NB) laryngeal endoscopy, the clinician usually positions the endoscope near the tissue for a correct inspection of possible vascular pattern alterations, indicative of laryngeal malignancies. The video is usually reviewed many times to refine the diagnosis, resulting in loss of time since the salient frames of the video are mixed with blurred, noisy, and redundant frames caused by the endoscope movements. The aim of this work is to provide to the clinician a unique larynx panorama, obtained through an automatic frame selection strategy to discard non-informative frames. Anisotropic diffusion filtering was exploited to lower the noise level while encouraging the selection of meaningful image features, and a feature-based stitching approach was carried out to generate the panorama. The frame selection strategy, tested on on six pathological NB endoscopic videos, was compared with standard strategies, as uniform and random sampling, showing higher performance of the subsequent stitching procedure, both visually, in terms of vascular structure preservation, and numerically, through a blur estimation metric.

In the last decades, major complications in surgery have emerged as a significant public health issue, so the practical implementation of safety measures to prevent injuries and deaths in different phases of surgery is required. The introduction of novel technologies in the operating theater, such as surgical robotic systems, opens new questions on how much and in which way the safety can be further improved. Computer Assisted Surgery (CAS), in combination with robotic systems, can greatly help in enhancing the surgeons' capabilities providing direct patient and process-specific support to surgeons with different degrees of experience. In particular, the application of Augmented Reality (AR) tools could represent a relevant step towards safer clinical procedures, improving the quality of healthcare. This chapter describes the main areas involved in an AR system, such as computer vision methods for identification of areas of interest, surgical scene description and safety warning methods. Recent advances in the field are also presented, providing as an example the Enhanced Vision System for Robotic Surgery (EnViSoRS): an AR system to provide assistance towards the protection of vessels from injury during the execution of surgical procedures with a commercial robotic surgical system.