Real-Time Dense Reconstruction of Tissue Surface From Stereo Optical Video

Zhou, Huajun; Jayender, Jagadeesan

doi:10.1109/tmi.2019.2927436

Cited by 40 publications

(24 citation statements)

References 44 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…( 23) and ( 24)) can obtain better results. [27], which were rendered by VTK. Note that the 3D model is not needed for 2D mosaicking.…”

Section: Methodsmentioning

confidence: 99%

“…The SLAM methods, which originated from the robotics navigation field, have been introduced for surgical navigation to extend the field of view of the laparoscope in real-time. For example, our previous work generated dense 3D model of the tissue surface from stereo laparoscopy videos by using SLAM [27]. Mahmoud et al proposed a monocular dense reconstruction method for tissue surfaces [28].…”

Section: A Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Real-time Nonrigid Mosaicking of Laparoscopy Images

Zhou,

Jayender

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D nonrigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.

show abstract

“…( 23) and ( 24)) can obtain better results. [27], which were rendered by VTK. Note that the 3D model is not needed for 2D mosaicking.…”

Section: Methodsmentioning

confidence: 99%

Section: A Related Workmentioning

confidence: 99%

Real-time Nonrigid Mosaicking of Laparoscopy Images

Zhou,

Jayender

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Readers may refer to Ref. [11] for more details on this stereo matching method. In our system, the initial template of the tissue surface is generated by the stereo matching method, then we track the deformation of the template by representing the non-rigid deformation with sparse control points on the template, and estimating the parameters of the control points to make the deformed template match the output of the GPU-based stereo matching method.…”

Section: Methodsmentioning

confidence: 99%

Real-Time Surface Deformation Recovery from Stereo Videos

Zhou¹,

Jayender²

2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Tissue deformation during the surgery may significantly decrease the accuracy of surgical navigation systems. In this paper, we propose an approach to estimate the deformation of tissue surface from stereo videos in real-time, which is capable of handling occlusion, smooth surface and fast deformation. We first use a stereo matching method to extract depth information from stereo video frames and generate the tissue template, and then estimate the deformation of the obtained template by minimizing ICP, ORB feature matching and as-rigid-as-possible (ARAP) costs. The main novelties are twofold: (1) Due to non-rigid deformation, feature matching outliers are difficult to be removed by traditional RANSAC methods; therefore we propose a novel 1-point RANSAC and reweighting method to preselect matching inliers, which handles smooth surfaces and fast deformations. (2) We propose a novel ARAP cost function based on dense connections between the control points to achieve better smoothing performance with limited number of iterations. Algorithms are designed and implemented for GPU parallel computing. Experiments on ex-and in vivo data showed that this approach works at an update rate of 15Hz with an accuracy of less than 2.5 mm on a NVIDIA Titan X GPU.

show abstract

“…Recently, Zhou et al [140,141] provided a comprehensive approach that includes enlarging the radius of the constraint of low texture regions, screening inliers from feature matching results with a histogram voting-based method, and tracking camera motion with a novel random sample consensus-based (RANSAC-based) perspective-n-point (PnP) algorithm. Luo et al [51] presented an AR-assisted navigation system for liver resection based on stereoscopy.…”

Section: ) Three-dimensional Surface Reconstructionmentioning

confidence: 99%

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Jin

Zhang

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Minimally invasive medicine has become mainstream because of its crucial clinical significance in providing a low risk of postoperative complications, limited blood loss, short postoperative recovery time, and small sizes of associated physiological tissue wounds. Endoscopic navigation systems comprise a research hot spot in medical science and technology and are an essential means to achieve precision medicine and improve surgical operation safety. As a core component in endoscopic navigation during minimally invasive surgery, endoscopes play a critical role in disease diagnosis and treatment. The development of endoscopic vision technologies has resulted in a renewed drive to further develop endoscopic navigation systems. Multiple endoscopic optical imaging modalities provide data sources for endoscopic vision technology, including white-light endoscopy, contrast-enhanced imaging and technologies involving magnified observation. Endoscopic vision is a specific application of computer vision involving the use of endoscopes that include instrument tracking, endoscopic view expansion, and suspicious lesion tracking in the application of endoscopic navigation. These techniques help surgeons or surgical robots locate instruments and lesions and expand the field of view of the endoscope. Although these technologies have been applied to various clinical and pre-clinical diagnoses and treatments, the use and combination of these advanced technologies in endoscopic navigation system for specific clinical requirements remains challenging. This review performs a broad survey of advanced endoscopic vision technologies and their application in endoscopic navigation systems. Finally, we discuss the challenges and future directions in implementing and developing endoscopic navigation systems.

show abstract

Real-Time Dense Reconstruction of Tissue Surface From Stereo Optical Video

Cited by 40 publications

References 44 publications

Real-time Nonrigid Mosaicking of Laparoscopy Images

Real-time Nonrigid Mosaicking of Laparoscopy Images

Real-Time Surface Deformation Recovery from Stereo Videos

The Future of Endoscopic Navigation: A Review of Advanced Endoscopic Vision Technology

Contact Info

Product

Resources

About