Combined video tracking and image-video registration for continuous bronchoscopic guidance

Abstract

Objective Three-dimensional (3D) computed-tomography (CT) images and bronchoscopy are commonly used tools for the assessment of lung cancer. Before bronchoscopy, the physician first examines a 3D CT chest image to select pertinent diagnostic sites and to ascertain possible routes through the airway tree leading to each site. Next, during bronchoscopy, the physician maneuvers the bronchoscope through the airways, basing navigation decisions on the live bronchoscopic video, to reach each diagnostic site. Unfortunately, no direct link exists between the 3D CT image data and bronchoscopic video. This makes bronchoscopy difficult to perform successfully. Existing methods for the image-based guidance of bronchoscopy, either only involve single-frame registration or operate at impractically slow frame rates. We describe a method that combines the 3D CT image data and bronchoscopic video to enable continuous bronchoscopic guidance. Methods The method interleaves periodic CT-video registration with bronchoscopic video motion tracking. It begins by using single-frame CT-video registration to register the "Real World" of the bronchoscopic video and the "Virtual World" of the CT-based endoluminal renderings. Next, the method uses an optical-flow-based approach to track bronchoscope movement for a fixed number of video frames and also simultaneously updates the virtual-world position. This process of registration and tracking repeats as the bronchoscope moves.Results The method operates robustly over a variety of phantom and human cases. It typically performs successfully for over 150 frames and through multiple airway generations. In our tests, the method runs at a rate of up to seven frames per second. We have integrated the method into a complete system for the image-based planning and guidance of bronchoscopy. Conclusion The method performs over an order of magnitude faster than previously proposed image-based bronchoscopy guidance methods. Software optimization and a modest hardware improvement could enable real-time performance.

Development and comparison of new hybrid motion tracking for bronchoscopic navigation

“…M8 corresponds to our proposed method utilizing Kalman filtering to get the unknown scale of the motion matrix obtained by epipolar geometry analysis. To evaluate the performance of the scale factor estimation, we also compared our method to two alternative approaches: (a) In M6, we simply assume the motion is constant and set the scale factor to 0.3, since the bronchoscope movement is of the order of about 0.3 mm/frame (at a video frame rate of 30 fps) (Rai et al, 2008); (b) In M7, we performed nonlinear optimization. Instead of optimizing the full six parameters in Eq.…”

“…However, feature-based methods for bronchoscope motion prediction seem promising for bronchoscope tracking. In Mori et al (2002) and Rai et al (2008), optical flow patterns compute bronchoscope motion between consecutive real bronchoscopic (RB) images as a pre-registration step for bronchoscopic navigation. Although good performance was shown, it remains difficult to get an accurate estimate for the insertion depth of the bronchoscope, or, in other words, for the magnitude of translation between successive frames.…”

Marker-Free Registration for Electromagnetic Navigation Bronchoscopy under Respiratory Motion

“…1 referring to the trachea, 2 to the trachea and left and right main bronchi, 3 to the trachea, left and right main bronchi, left and right upper lobe bronchi, left lower lobe bronchus and right truncus intermedius, and so on. We set the speed of the bronchoscope to 10 mm/s, as this is a good approximation of the average speed during bronchoscopy [4]. The sampling rate was set to 40 Hz, according to the frequencies of typical EMT systems such as the NDI Aurora or the Ascension 3D Guidance medSAFE.…”

“…Image based techniques try to register virtual camera images generated from preinterventional computed tomography data to the real images acquired by the bronchoscope camera [3,4], which can be time-consuming and fail to continuously track the camera motion. Continuous and real-time electromagnetic tracking (EMT) of a small sensor coil attached to the bronchoscope tip can resolve these issues.…”

Robust camera localisation with depth reconstruction for bronchoscopic navigation

“…Higgins et al [9,20] tracked the 3D motion of the bronchoscope by estimating optical flow from video images and then using the tracked 3D trajectory to assist localisation in the 3D CT virtual world. Nagao et al [17] employed the principle of Kalman filter to predict the motion of the bronchoscope.…”