Rendering-based video-CT registration with physical constraints for image-guided endoscopic sinus surgery

Otake, Yoshito; Léonard, Simon; Reiter, Austin; Rajan, Purnima; Siewerdsen, Jeffrey H.; Gallia, Gary L.; Ishii, Masaru; Taylor, Russell H.; Hager, Gregory D.

doi:10.1117/12.2081732

Cited by 9 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In endoscope-CT registration, combinations of different intensity-based schemes such as cross-correlation, squared intensity difference, pattern intensity, normalised and gradient mutual information have shown promising results 40,41 . Similarly, feature-based schemes involving natural landmarks, contour based feature points, iterative closest point and k-means clustering have also been exploited [42][43][44] . The main challenge of our system was the low similarity between the multi-modal images.…”

Section: Discussionmentioning

confidence: 99%

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Hussain¹,

Lalande²,

Marroquin³

et al. 2020

Sci Rep

View full text Add to dashboard Cite

the aim of the study was to develop and assess the performance of a video-based augmented reality system, combining preoperative computed tomography (ct) and real-time microscopic video, as the first crucial step to keyhole middle ear procedures through a tympanic membrane puncture. Six different artificial human temporal bones were included in this prospective study. Six stainless steel fiducial markers were glued on the periphery of the eardrum, and a high-resolution CT-scan of the temporal bone was obtained. Virtual endoscopy of the middle ear based on this CT-scan was conducted on Osirix software. Virtual endoscopy image was registered to the microscope-based video of the intact tympanic membrane based on fiducial markers and a homography transformation was applied during microscope movements. These movements were tracked using Speeded-Up Robust Features (SURF) method. Simultaneously, a micro-surgical instrument was identified and tracked using a Kalman filter. The 3D position of the instrument was extracted by solving a three-point perspective framework. For evaluation, the instrument was introduced through the tympanic membrane and ink droplets were injected on three middle ear structures. An average initial registration accuracy of 0.21 ± 0.10 mm (n = 3) was achieved with a slow propagation error during tracking (0.04 ± 0.07 mm). The estimated surgical instrument tip position error was 0.33 ± 0.22 mm. The target structures' localization accuracy was 0.52 ± 0.15 mm. The submillimetric accuracy of our system without tracker is compatible with ear surgery.

show abstract

Section: Discussionmentioning

confidence: 99%

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Hussain¹,

Lalande²,

Marroquin³

et al. 2020

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Tracking patient motion may be necessary in some contexts that demand surgical precision or when surgery results in extensive surface motion, such as in external eye surgery [74]. At a granular level, tracking changes in tissue during surgery is more challenging with deformable soft tissue [75], [76] than with rigid anatomical structures such as the paranasal sinuses [77]. Beyond the patient, several methods to estimate motion or changes in pose of surgical instruments using video images and/or kinematics have been developed [78], for example, in minimally invasive surgery [79], [80], open surgery [81], microsurgery [82], endoscopy [83], bronchoscopy [84], and laser surgery [85].…”

Section: Examples and Potential Applicationsmentioning

confidence: 99%

Surgical data science: the new knowledge domain

Vedula

Hager

2017

Innovative Surgical Sciences

View full text Add to dashboard Cite

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care with the goal of maximizing quality and value of care. While innovations in diagnostic and therapeutic technologies have driven past improvements in quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytic techniques, and translation or integration of research findings into patient care. We foresee the emergence of Surgical/Interventional Data Science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model and quantify the pathways or processes within the context of patient health states or outcomes, and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data is pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care including prevention, diagnosis, intervention, or post-operative recovery. Existing literature already provides preliminary results suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from pre-, intra-, and post-operative contexts, how it could support intra-operative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

show abstract

“…The maximum registration error for IG-FESS that is most commonly found in the literature is 2 mm 7, 8 and accuracy of less than 1.5 mm have been reported for modern navigation systems. 9 Recently, an image-based registration method achieved reprojection error 0.7 mm 10 but this methods require an initial registration to function.…”

Section: Introductionmentioning

confidence: 95%

Image-based navigation for functional endoscopic sinus surgery using structure from motion

et al. 2016

View full text Add to dashboard Cite

Functional Endoscopic Sinus Surgery (FESS) is a challenging procedure for otolaryngologists and is the main surgical approach for treating chronic sinusitis, to remove nasal polyps and open up passageways. To reach the source of the problem and to ultimately remove it, the surgeons must often remove several layers of cartilage and tissues. Often, the cartilage occludes or is within a few millimeters of critical anatomical structures such as nerves, arteries and ducts. To make FESS safer, surgeons use navigation systems that register a patient to his/her CT scan and track the position of the tools inside the patient. Current navigation systems, however, suffer from tracking errors greater than 1 mm, which is large when compared to the scale of the sinus cavities, and errors of this magnitude prevent from accurately overlaying virtual structures on the endoscope images. In this paper, we present a method to facilitate this task by 1) registering endoscopic images to CT data and 2) overlaying areas of interests on endoscope images to improve the safety of the procedure. First, our system uses structure from motion (SfM) to generate a small cloud of 3D points from a short video sequence. Then, it uses iterative closest point (ICP) algorithm to register the points to a 3D mesh that represents a section of a patients sinuses. The scale of the point cloud is approximated by measuring the magnitude of the endoscope’s motion during the sequence. We have recorded several video sequences from five patients and, given a reasonable initial registration estimate, our results demonstrate an average registration error of 1.21 mm when the endoscope is viewing erectile tissues and an average registration error of 0.91 mm when the endoscope is viewing non-erectile tissues. Our implementation SfM + ICP can execute in less than 7 seconds and can use as few as 15 frames (0.5 second of video). Future work will involve clinical validation of our results and strengthening the robustness to initial guesses and erectile tissues.

show abstract

Rendering-based video-CT registration with physical constraints for image-guided endoscopic sinus surgery

Cited by 9 publications

References 14 publications

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Surgical data science: the new knowledge domain

Image-based navigation for functional endoscopic sinus surgery using structure from motion

Contact Info

Product

Resources

About