Automated surgical workflow analysis and understanding can assist surgeons to standardize procedures and enhance post-surgical assessment and indexing, as well as, interventional monitoring. Computerassisted interventional (CAI) systems based on video can perform workflow estimation through surgical instruments' recognition while linking them to an ontology of procedural phases. In this work, we adopt a deep learning paradigm to detect surgical instruments in cataract surgery videos which in turn feed a surgical phase inference recurrent network that encodes temporal aspects of phase steps within the phase classification. Our models present comparable to state-of-the-art results for surgical tool detection and phase recognition with accuracies of 99 and 78% respectively.

Abstract-The field of robotic surgery increasingly advances towards highly articulated and continuum robots, requiring new kinematic strategies to enable users to perform dexterous manipulation in confined workspaces. This development is driven by surgical interventions accessing the surgical workspace through natural orifices such as the mouth or the anus. Due to the long and narrow nature of these access pathways, external triangulation at the fulcrum point is very limited or absent, which makes introducing multiple degrees of freedom at the distal end of the instrument necessary. Additionally, high force and miniaturization requirements make the control of such instruments particularly challenging. This paper presents the kinematic considerations needed to effectively manipulate these novel instruments and allow their dexterous control in confined spaces. A non-linear calibration model is further used to map joint to actuator space and improve significantly the precision of the instrument's motion. The effectiveness of the presented approach is quantified with bench tests, and the usability of the system is assessed by three user studies simulating the requirements of a realistic surgical task.

Abstract-This paper introduces a single-port robotic platform for Transanal Endoscopic Micro-Surgery (TEMS). Two robotically controlled articulated surgical instruments are inserted via a transanal approach to perform submucosal or fullthickness dissection. This system is intended to replace the conventional TEMS approach that uses manual laparoscopic instruments. The new system is based on master-slave robotically controlled tele-manipulation. The slave robot comprises a support arm that is mounted on the operating table, supporting a surgical port and a robotic platform that drives the surgical instruments. The master console includes a pair of haptic devices, as well as a 3D display showing the live video stream of a stereo endoscope inserted through the surgical port. The surgical instrumentation consists of energy delivery devices, graspers and needle drivers allowing a full TEMS procedure to be performed. Results from benchtop tests, ex-vivo animal tissue evaluation and in-vivo studies demonstrate the clinical advantage of the proposed system.

Abstract-Robotic-assisted Minimally Invasive Surgery (RMIS) can benefit from the automation of common, repetitive or well-defined but ergonomically difficult tasks. One such task is the scanning of a pick-up endomicroscopy probe over a complex, undulating tissue surface in order to enhance the effective field-of-view through video mosaicing. In this paper, the da Vinci ® surgical robot, through the dVRK framework, is used for autonomous scanning and 2D mosaicing over a user-defined region of interest. To achieve the level of precision required for high quality mosaic generation, which relies on sufficient overlap between consecutive image frames, visual servoing is performed using a combination of a tracking marker attached to the probe and the endomicroscopy images themselves. The resulting sub-millimetre accuracy of the probe motion allows for the generation of large endomicroscopy mosaics with minimal intervention from the surgeon. It also allows the probe to be maintained in an orientation perpendicular to the local tissue surface, providing optimal imaging results. Images are streamed from the endomicroscope and overlaid live onto the surgeons view, while 2D mosaics are generated in real-time, and fused into a 3D stereo reconstruction of the surgical scene, thus providing intuitive visualisation and fusion of the multiscale images. The system therefore offers significant potential to enhance surgical procedures, by providing the operator with cellular-scale information over a larger area than could typically be achieved by manual scanning.

Purpose: For effective tumour margin definition for cancer surgery, there is an increasing demand for the development of real-time intraoperative tissue biopsy techniques. Recent advances in miniaturized biophotonics probes have permitted the development of endomicroscopy techniques that are clinically attractive. With these approaches, cellular-level imaging can be achieved through millimetre-scale flexible probes, and be performed in real-time, in vivo and in situ. Due to the limited field-of-view and flexibility of these probes, however, large area tissue coverage for acquiring histology-like images over complex three-dimensional surfaces is challenging. This is particularly the case because current surgical robots, such as the Da Vinci®, lack haptic feedback, making it difficult to maintain optimum tissue contact when these probes are deployed in vivo.Methods: This paper proposes a simple force-controlled pick-up probe that can be integrated with the Da Vinci instruments for intraoperative endomicroscopy imaging. The device uses a new lowfriction air bearing with adaptive axial force control to maintain constant contact between the tissue and the imaging probe, facilitating microscopy scans over complex surfaces. Detailed ex vivo user experiments have been conducted to demonstrate the effectiveness of the technique.Results: The adaptive probe mount could achieve consistent low magnitude probe-sample contact forces compared to a rigid mount. In the user study, the adaptive probe combined with a high frame rate endomicroscopy system allowed larger mosaics to be generated over curved surfaces.Conclusion: The device can improve the performance of large area mosaicking over complex 3D surfaces with improved handling and intraoperative control.

This approach presents an important alternative to current histology techniques, significantly reducing the tissue assessment time, while simultaneously providing the capability to mark and ablate suspicious areas intraoperatively.

Computer-assisted interventions (CAI) aim to increase the effectiveness, precision and repeatability of procedures to improve surgical outcomes. The presence and motion of surgical tools is a key information input for CAI surgical phase recognition algorithms. Vision-based tool detection and recognition approaches are an attractive solution and can be designed to take advantage of the powerful deep learning paradigm that is rapidly advancing image recognition and classification. The challenge for such algorithms is the availability and quality of labelled data used for training. In this Letter, surgical simulation is used to train tool detection and segmentation based on deep convolutional neural networks and generative adversarial networks. The authors experiment with two network architectures for image segmentation in tool classes commonly encountered during cataract surgery. A commercially-available simulator is used to create a simulated cataract dataset for training models prior to performing transfer learning on real surgical data. To the best of authors’ knowledge, this is the first attempt to train deep learning models for surgical instrument detection on simulated data while demonstrating promising results to generalise on real data. Results indicate that simulated data does have some potential for training advanced classification methods for CAI systems.

ABSTRACT² We propose a learning-based controller to enable autonomous execution of the eFAST scanning by a lightweight robotic manipulator according to expert demonstrations. The benefits of this approach are two-fold. Firstly, the automatically acquired USS images can be sent to the expert radiologist from a remote location without the need for complex robotic teleoperation. Secondly, the application of learning by demonstration alleviates the complexity of robotic programming and allows extracting operator-specific knowledge IN SITU in a natural and intuitive way. The provision of incorporating force information can further improve the versatility of the system, allowing easy adaptation to different dynamic environments.