Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis

Le, Hanh N. D.; Nguyen, Hieu; Wang, Zhaoyang; Opfermann, Justin D.; Léonard, Simon; Krieger, Axel; Kang, Jin U.

doi:10.1117/1.jbo.23.5.056009

Cited by 21 publications

(11 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…1 and Movie 1. The system consists of one KUKA LBR Med robot with a motorized Endo 360 suturing tool ( 37 ) for robotic suturing and a second KUKA LBR Med robot that carries an endoscopic dual-camera system consisting of an NIR camera and a 3D mono color endoscope ( 38 ). The camera system allows the STAR to autonomously track biocompatible NIR markers ( 39 ) on the tissue and reconstruct the 3D surface of the tissue (used for suture planning).…”

Section: Resultsmentioning

confidence: 99%

Autonomous robotic laparoscopic surgery for intestinal anastomosis

et al. 2022

View full text Add to dashboard Cite

Autonomous robotic surgery has the potential to provide efficacy, safety, and consistency independent of individual surgeon’s skill and experience. Autonomous anastomosis is a challenging soft-tissue surgery task because it requires intricate imaging, tissue tracking, and surgical planning techniques, as well as a precise execution via highly adaptable control strategies often in unstructured and deformable environments. In the laparoscopic setting, such surgeries are even more challenging because of the need for high maneuverability and repeatability under motion and vision constraints. Here we describe an enhanced autonomous strategy for laparoscopic soft tissue surgery and demonstrate robotic laparoscopic small bowel anastomosis in phantom and in vivo intestinal tissues. This enhanced autonomous strategy allows the operator to select among autonomously generated surgical plans and the robot executes a wide range of tasks independently. We then use our enhanced autonomous strategy to perform in vivo autonomous robotic laparoscopic surgery for intestinal anastomosis on porcine models over a 1-week survival period. We compared the anastomosis quality criteria—including needle placement corrections, suture spacing, suture bite size, completion time, lumen patency, and leak pressure—of the developed autonomous system, manual laparoscopic surgery, and robot-assisted surgery (RAS). Data from a phantom model indicate that our system outperforms expert surgeons’ manual technique and RAS technique in terms of consistency and accuracy. This was also replicated in the in vivo model. These results demonstrate that surgical robots exhibiting high levels of autonomy have the potential to improve consistency, patient outcomes, and access to a standard surgical technique.

show abstract

Section: Resultsmentioning

confidence: 99%

Autonomous robotic laparoscopic surgery for intestinal anastomosis

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In order to cope with the phase-unwrapping difficulty encountered in the cases of complex shapes and geometric discontinuities, a scheme of using multi-frequency fringe patterns is often employed in practice. The corresponding unwrapped phase distributions can be calculated from [ 43 , 44 , 45 ]:

where

is the unwrapped phase; i indicates the i th fringe-frequency pattern with

, and n is the number of fringe frequencies; INT represents the function of rounding to the nearest integer;

is the number of fringes in the i th projection pattern, with

; and

is satisfied for

. The ratio between two adjacent fringe frequencies

is normally smaller or equal to 5 to reduce the noise effect and ensure the reliability of the algorithm.…”

Section: Methodsmentioning

confidence: 99%

Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks

Nguyen

Wang

2020

Sensors

Self Cite

View full text Add to dashboard Cite

Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.

show abstract

“…Another sector of research investigates the use of specialized hardware for surface reconstruction. Over the years, several research groups proposed the use of structured-light projectors to enhance shape estimation in endoscopic or laparoscopic imaging systems [42]- [46]. As a remarkable example, Maurice et al presented a system capable of integrating light patterns into a full HD endoscopic stream at 25 Hz.…”

Section: B Structured Light and Time-of-flight Sensingmentioning

confidence: 99%

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

Caccianiga¹,

Kuchenbecker²

2022

Proceedings of the 14th Hamlyn Symposium on Medical Robotics 2022

View full text Add to dashboard Cite

Technical innovations in sensing and computation are quickly advancing the Ô¨Åeld of computer-integrated surgery. From one side, spectral imaging and biophoton- ics are strongly contributing to intraoperative diagnostics and decision making. Simultaneously, learning-based algorithms are reshaping the concept of assistance and prediction in surgery. In this fast-evolving panorama, we strongly believe there is still a need for robust geometric reconstruction of the surgical Ô¨Åeld whether the goal is traditional surgical assistance or partial or full autonomy. 3D reconstruction in surgery has been investigated almost only in the space of mono and stereoscopic visual imaging because surgeons always view the proce- dure through a clinical endoscope. Compared to using traditional computer vision, deep learning has made signiÔ¨Åcant progress in creating high-quality 3D recon- structions and dense maps from such data streams, especially for monocular simultaneous localization and mapping (SLAM) [1]. The main limitations are linked to reliability, generalization, and computational cost. Meanwhile, lidar (light detection and ranging) has greatly expanded in use, especially in SLAM for robotics, terrestrial vehicles, and drones. Lidar sensors explicitly measure the depth Ô¨Åeld rather than inferring it from camera images. The technology is evolving quickly thanks to the upsurge of mixed and augmented reality in consumer mobile devices [2]: high-resolution, short- range, miniaturized lidar sensors are expected soon. In parallel to these developments, the concept of multiple-viewpoint surgical imaging was proposed in the early 2010‚Äôs in the context of magnetic actuation and micro-invasive surgery [3]. In routine clinical practice, however, the use of multiple trans-abdominal cannulae still limits the kinematics of the camera and instruments to have a Ô¨Åxed pivot point at the body wall. For this reason, here we propose an approach in which each surgical cannula can potentially hold a miniature lidar. We envision that exploring this powerful sensing tech- nology and enabling multi-viewpoint imaging without disrupting the current surgical workÔ¨Çow will yield far more accurate and complete 3D reconstructions of the surgical Ô¨Åeld, which opens new opportunities for the future of computer-integrated surgery.

show abstract

Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis

Cited by 21 publications

References 44 publications

Autonomous robotic laparoscopic surgery for intestinal anastomosis

Autonomous robotic laparoscopic surgery for intestinal anastomosis

Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks

Dense 3D Reconstruction Through Lidar: A New Perspective on Computer-Integrated Surgery

Contact Info

Product

Resources

About