Surgical Navigation by Autostereoscopic Image Overlay of Integral Videography

Abstract

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.

Advanced Medical Displays: A Literature Review of Augmented Reality

“…In 2002 Liao et al [36], [37] proposed a medical AR window based on integral videography that could handle dynamic scenes. The authors realized the system for a navigation scenario, in which the position of an instrument was supposed to be visualized in the scene.…”

High Quality Autostereoscopic Surgical Display Using Anti-aliased Integral Videography Imaging

“…7b). Intra-operatively, IV image overlay technique w ith corresponding image registration can help with the navigation by providing a broader view of the operation field [7]. In combination with robotic and surgical instrument, it even can supply guidance by pre-defining the path of a biopsy needle or by preventing the surgical instruments from moving into critical regions.…”

“…IP and IV record and reproduce 3-D imagesu sing a micro convex lens array and photographic film (or a flat display). Ahigh-resolutionm ulti-projectionIV display system [6] and a surgical navigation system by IV image overlay [7] are introduced for image-guided surgery. With additional improvements in the display, these systems will increase the surgical accuracy and reduce invasiveness.…”

“…Despite IV's many advantages that have been proven in bothf easibility studies and clinical applications [6][7][8], the quality of its spatial image has thus far been poor. Furthermore, a high quality 3-D visualization system must be developed to integrate capabilities forpre-surgical/intra-operative image guidance.…”

Robust and Accurate Algorithm for Wearable Stereoscopic Augmented Reality with Three Indistinguishable Markers

“…Alternative and promising approaches based on integral imaging (II) technology have been proposed [13,14]. II displays use a set of 2D elemental images from different perspectives to generate a full-parallax 3D visualization.…”