Construction of a multimodal CT-video chest model

Abstract

Bronchoscopy enables a number of minimally invasive chest procedures for diseases such as lung cancer and asthma. For example, using the bronchoscope's continuous video stream as a guide, a physician can navigate through the lung airways to examine general airway health, collect tissue samples, or administer a disease treatment. In addition, physicians can now use new image-guided intervention (IGI) systems, which draw upon both three-dimensional (3D) multi-detector computed tomography (MDCT) chest scans and bronchoscopic video, to assist with bronchoscope navigation. Unfortunately, little use is made of the acquired video stream, a potentially invaluable source of information. In addition, little effort has been made to link the bronchoscopic video stream to the detailed anatomical information given by a patient's 3D MDCT chest scan. We propose a method for constructing a multimodal CT-video model of the chest. After automatically computing a patient's 3D MDCT-based airway-tree model, the method next parses the available video data to generate a positional linkage between a sparse set of key video frames and airway path locations. Next, a fusion/mapping of the video's color mucosal information and MDCT-based endoluminal surfaces is performed. This results in the final multimodal CT-video chest model. The data structure constituting the model provides a history of those airway locations visited during bronchoscopy. It also provides for quick visual access to relevant sections of the airway wall by condensing large portions of endoscopic video into representative frames containing important structural and textural information. When examined with a set of interactive visualization tools, the resulting fused data structure provides a rich multimodal data source. We demonstrate the potential of the multimodal model with both phantom and human data.

Method for endobronchial video parsing

“…In prior work, we presented the concept of a multi-modal CT-video chest model which fuses summarized endobronchial video sequences with a CT-derived airway-tree model. 4 This effort depended on manually selected key frames and reported an average reduction of example video sequences of 65%. Our current method represents a major step forward from our earlier work toward a fully automatic method for fusing endobronchial video with a CT-derived airway-tree model.…”

“…Finally, based on the condition number of H, if the number of quality feature matches between the two frames is sufficient, then we consider the frames to be matched and add the latter frame to current shot S i . 4 Otherwise, subsequent frames are compared against the current frame until a search neighborhood range is exceeded. This process continues for a new shot, with S i+1 initialized with the frame following the last frame of S i .…”

“…Additionally, there currently exists no convenient meansother than physician notes and memory -to incorporate recorded bronchoscopic video into additional follow-up procedures often required in asthma treatment or cancer assessment. [2][3][4] As in related endoscopy domains (e.g., colonoscopy, wireless-capsule endoscopy), a lack of mature and robust video-analysis methods for parsing and summarizing endoscopic video prevents the full utilization of this rich data source. [5][6][7] To address this need, we propose a fully automatic method for parsing an endobronchial video sequence.…”

Generative Localization With Uncertainty Estimation Through Video-CT Data for Bronchoscopic Biopsy

“…Furthermore, rather than using the time consuming pixel-wise similarity measures, more efficient registration approaches based on airway lumen feature matching have been proposed in [17][16][4] to perform realtime bronchoscopic localisation. In addition, a feature-based visual SLAM [18] is also investigated for localisation in the endobronchial environment. However, the visual odometry approach is prone to insufficient visual features such as SIFT or ORB for tracking.…”

Automated frame selection process for high-resolution microendoscopy

“…Recent clinical studies of high-resolution fiber-optic microendoscopy have demonstrated that this method can be used to detect neoplastic lesions in patients with oral squamous cell carcinoma 6 and Barrett's esophagus. Automated frame selection algorithms and procedures have been reported for laparoscopic videos, 9 colonoscopy videos, 10 capsule endoscopy videos, [11][12][13][14][15] cystoscopy videos, 16 angiography videos, 17 bronchoscopic videos, 18 larynx endoscopy videos, 19 and retinal videos. The field of view is typically 0.5 to 1.0 mm in diameter with a lateral resolution of about 4 μm.…”