Implementation, calibration and accuracy testing of an image-enhanced endoscopy system

Abstract

This paper presents a new method for image-guided surgery called image-enhanced endoscopy. Registered real and virtual endoscopic images (perspective volume renderings generated from the same view as the endoscope camera using a preoperative image) are displayed simultaneously; when combined with the ability to vary tissue transparency in the virtual images, this provides surgeons with the ability to see beyond visible surfaces and, thus, provides additional exposure during surgery. A mount with four photoreflective spheres is rigidly attached to the endoscope and its position and orientation is tracked using an optical position sensor. Generation of virtual images that are accurately registered to the real endoscopic images requires calibration of the tracked endoscope. The calibration process determines intrinsic parameters (that represent the projection of three-dimensional points onto the two-dimensional endoscope camera imaging plane) and extrinsic parameters (that represent the transformation from the coordinate system of the tracker mount attached to the endoscope to the coordinate system of the endoscope camera), and determines radial lens distortion. The calibration routine is fast, automatic, accurate and reliable, and is insensitive to rotational orientation of the endoscope. The routine automatically detects, localizes, and identifies dots in a video image snapshot of the calibration target grid and determines the calibration parameters from the sets of known physical coordinates and localized image coordinates of the target grid dots. Using nonlinear lens-distortion correction, which can be performed at real-time rates (30 frames per second), the mean projection error is less than 0.5 mm at distances up to 25 mm from the endoscope tip, and less than 1.0 mm up to 45 mm. Experimental measurements and point-based registration error theory show that the tracking error is about 0.5-0.7 mm at the tip of the endoscope and less than 0.9 mm for all points in the field of view of the endoscope camera at a distance of up to 65 mm from the tip. It is probable that much of the projection error is due to endoscope tracking error rather than calibration error. Two examples of clinical applications are presented to illustrate the usefulness of image-enhanced endoscopy. This method is a useful addition to conventional image-guidance systems, which generally show only the position of the tip (and sometimes the orientation) of a surgical instrument or probe on reformatted image slices.

Fusion of video and motion data: engineering tasks and clinical applicability

“…In this case there is a chance to move faster with new applications of real clinical value and usability. Procedure, similar to the presented one in this paper, for tracking the location of endoscopic image relative to CT volume is described in 16 …”

Fully Automatic Endoscope Calibration for Intraoperative Use

“…[5] used a "center dot" to determine the orientation of the dot grid. The center dot detection is based on the relative radius (classification in large dots vs. small dots), which could be problematic with severe distortions.…”

Laparoscopic Virtual Mirror for Understanding Vessel Structure Evaluation Study by Twelve Surgeons

“…Endoscope augmentation was proposed for different medical applications like brain surgery [9], liver surgery [7,3,8], transbronchial biopsy [2] and cardiac surgery [5] to support different procedures in the surgical workflow such as port placement and navigation of instruments inside the patient. Fuchs et al introduced a system for laparoscope surgery displaying data with a head mounted display [4] instead on an external monitor.…”