No abstract

Abstract-Computer vision and robotics are being increasingly applied in medical interventions. Especially in interventions where extreme precision is required they could make a difference. One such application is robot-assisted retinal microsurgery. In recent works, such interventions are conducted under a stereo-microscope, and with a robot-controlled surgical tool. The complementarity of computer vision and robotics has however not yet been fully exploited. In order to improve the robot control we are interested in 3D reconstruction of the anatomy and in automatic tool localization using a stereo microscope. In this paper, we solve this problem for the first time using a single pipeline, starting from uncalibrated cameras to reach metric 3D reconstruction and registration, in retinal microsurgery. The key ingredients of our method are: (a) surgical tool landmark detection, and (b) 3D reconstruction with the stereo microscope, using the detected landmarks. To address the former, we propose a novel deep learning method that detects and recognizes keypoints in high definition images at higher than real-time speed. We use the detected 2D keypoints along with their corresponding 3D coordinates obtained from the robot sensors to calibrate the stereo microscope using an affine projection model. We design an online 3D reconstruction pipeline that makes use of smoothness constraints and performs robot-to-camera registration. The entire pipeline is extensively validated on open-sky porcine eye sequences. Quantitative and qualitative results are presented for all steps.

If citing, it is advised that you check and use the publisher's definitive version for pagination, volume/issue, and date of publication details. And where the final published version is provided on the Research Portal, if citing you are again advised to check the publisher's website for any subsequent corrections.

Retinal Vein Occlusion is a common retinal vascular disorder which can cause severe loss of vision. Retinal vein cannulation followed by injection of an anti-coagulant into the affected vein is a promising treatment. However, given the scale and fragility of the surgical workfield, this procedure is considered too high-risk to perform manually. A first successful robot-assisted procedure has been demonstrated. Even though successful, the procedure remains extremely challenging. This paper aims at providing a solution for the limited perception of instrument-tissue interaction forces as well as depth estimation during retinal vein cannulation. The development of a novel combined force and distance sensing cannulation needle relying on Fiber Bragg grating (FBG) and Optical Coherence Tomography (OCT) A-scan technology is reported. The design, the manufacturing process, the calibration method, and the experimental characterization of the produced sensor are discussed. The functionality of the combined sensing modalities and the real-time distance estimation algorithm are validated respectively on in-vitro and ex-vivo models.

Minimally invasive surgery is now a well established field in surgery but continuous efforts are made to reduce invasiveness even further. This paper proposes a novel concept of small-diameter multi-arm instrument for Single-Port Access Surgery. The concept introduces a novel combination of backbone and actuation principles in a macro-micro fashion to achieve an excellent decoupling of the triangulation platform (macro) and of the end-effectors (micro). Concentric tube robots are used for the triangulation platform, while compliant fluidic-actuated bending segments are used as end-effectors. The fluidic actuation is advantageous as it minimally interferes with the triangulation platform. The triangulation platform on the other hand provides a stable base for the end-effectors such that large distal actuation bandwidth can be achieved. A specific embodiment for Spina Bifida repair is developed and proposed. The surgical and technical requirements as well as the mechanical design are presented in details. A first prototype is built and characterization experiments are conducted to evaluate its performance.

Catheters are increasingly being used to tackle problems in the cardiovascular system. However, positioning precision of the catheter tip is negatively affected by hysteresis. To ensure tissue damage due to imprecise positioning is avoided, hysteresis is to be understood and compensated for. This work investigates the feasibility to model hysteresis with a Long Short-Term Memory (LSTM) network. A bench-top setup containing a catheter distal segment was developed for model evaluation. The LSTM was first tested using four groups of test datasets containing diverse patterns. To compare with the LSTM, a Deadband Rate-Dependent Prandtl-Ishlinskii (DRDPI) model and a Support Vector Regression (SVR) model were established. The results demonstrated that the LSTM is capable of predicting the tip bending angle with sub-degree precision. The LSTM outperformed the DRDPI model and the SVR model by 60.1% and 36.0%, respectively, in arbitrarily varying signals. Next, the LSTM was further validated in a 3D reconstruction experiment using Forward-Looking Optical Coherence Tomography (FL-OCT). The results revealed that the LSTM was able to accurately reconstruct the environment with a reconstruction error below 0.25 mm. Overall, the proposed LSTM enabled precise free-space control of a robotic catheter in the presence of severe hysteresis. The LSTM predicted the catheter tip response precisely based on proximal input pressure, minimizing the need to install sensors at the catheter tip for localization.

The results demonstrate the feasibility of deploying a combined sensing instrument in an in vivo setting. The performed study provides a foundation for further work on real-time local modelling of the surgical scene. This paper provides initial insights; however, additional processing remains necessary.

No abstract