The feasibility of endoscopy-CT image registration in the head and neck without prospective endoscope tracking

Abstract

PurposeEndoscopic examinations are frequently-used procedures for patients with head and neck cancer undergoing radiotherapy, but radiation treatment plans are created on computed tomography (CT) scans. Image registration between endoscopic video and CT could be used to improve treatment planning and analysis of radiation-related normal tissue toxicity. The purpose of this study was to explore the feasibility of endoscopy-CT image registration without prospective physical tracking of the endoscope during the examination.MethodsA novel registration technique called Location Search was developed. This technique uses physical constraints on the endoscope’s view direction to search for the virtual endoscope coordinates that maximize the similarity between the endoscopic video frame and the virtual endoscopic image. Its performance was tested on phantom and patient images and compared to an established registration technique, Frame-To-Frame Tracking.ResultsIn phantoms, Location Search had average registration errors of 0.55 ± 0.60 cm for point measurements and 0.29 ± 0.15 cm for object surface measurements. Frame-To-Frame Tracking achieved similar results on some frames, but it failed on others due to the virtual endoscope becoming lost. This weakness was more pronounced in patients, where Frame-To-Frame tracking could not make it through the nasal cavity. On successful patient video frames, Location Search was able to find endoscope positions with an average distance of 0.98 ± 0.53 cm away from the ground truth positions. However, it failed on many frames due to false similarity matches caused by anatomical structural differences between the endoscopic video and the virtual endoscopic images.ConclusionsEndoscopy-CT image registration without prospective physical tracking of the endoscope is possible, but more development is required to achieve an accuracy suitable for clinical translation.

Efficacy of Endoscopic Evaluation of Acute Radiation Esophagitis during Chemoradiotherapy with Proton Beam Therapy Boost for Esophageal Cancer

“…In addition, by providing prior understanding of which patients are at higher risk during dose escalation (information that can be shared with the patients), the FARE grade can provide the basis for more careful observations and rapid management of adverse events. Furthermore, if spatial localization of esophagitis is possible with recently developed endoscopy-CT registration system [29], it might be able to correlate this information into the treatment planning process in the future.…”

Combining Deep Learning with Geometric Features for Image based Localization in the Gastrointestinal Tract

“…Unlike researches [11], [10], [1], [9] bridging the prior 3D image (CT scan) with 2D image-based observation, our work aims at imagebased localization in the scenario with global pose retrieved from some sources like Optitrack [8], robot arm [3] or Electro-magnetic sensor [5]. In the case of CT scan as the training data in [11], [10], [1], [9], the image poses labels can be retrieved offline with similarity based automatic image to CT registration [9], [10], [6], or from semi-automatic pose labeling, because our work only requires small training dataset.…”

“…However, due to the difficulties in multiple sensor fusion or high cost, image-based localization is still popular in CAS and/or screening applications for medical practitioners. Various techniques like image to CT registration [6], Simultaneous Localization And Mapping (SLAM) [7], Visual Odometry (VO) [8] and deep learning (DL) [3] are analyzed. Moreover, image-based localization The state-of-art approaches include feature based [9], [10], [11], [7], [12], DL based methods [1], [8] and fusing DL with geometric [8].…”

A novel intrarenal multimodal 2D/3D registration algorithm and preliminary phantom study

“…The results show that the contour extraction method in this paper achieves acceptable $$KP{I}_{{\Psi}}$$ values (all less than 0.5) and performs well on phantoms. The contour extraction error of the real image is higher than that of the phantom, but it is still below 0.5, and the extraction results using the cycleGAN network is better than that directly from the original image. We evaluated the performance of the proposed method and compared it with five competing methods used in 2D/3D registration for endoscope: NMI, 20 $$M{I}_{grad}$$, 23 RANSAC‐PnP, 19 MS‐DSSM 25 and CycleGAN‐Depth 15 . The Powell algorithm was used to optimize the competing method, and the initial value of the optimization was set to μ 0 .…”

“…We evaluated the performance of the proposed method and compared it with five competing methods used in 2D/3D registration for endoscope: NMI, 20 $$M{I}_{grad}$$, 23 RANSAC‐PnP, 19 MS‐DSSM 25 and CycleGAN‐Depth 15 . The Powell algorithm was used to optimize the competing method, and the initial value of the optimization was set to μ 0 .…”