Projector-Based Augmented Reality for Intuitive Intraoperative Guidance in Image-Guided 3D Interstitial Brachytherapy

Krempien, Robert; Hoppe, Harald; Kahrs, Lüder A.; Daeuber, Sascha; Schorr, Oliver; Eggers, Georg; Bischof, Marc; Münter, Marc W.; Debus, Jürgen; Harms, Wolfgang

doi:10.1016/j.ijrobp.2007.10.048

Cited by 49 publications

(38 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is a prerequisite of our approach that planning information is either projected onto the liver surface during open liver surgery using a light projector as described in [14][15][16], or rather superimposed with the images from a laparoscopic camera [11][12][13]. Since the surgical view is augmented with planning models, we expect a reduction of cognitive demands for the surgeon, concerning distracting comparisons of spatially separated views during the intervention.…”

Section: Methodsmentioning

confidence: 99%

“…Projector-based AR represents an interesting way to support surgical decisions: Krempien et al [14] and Riechmann et al [15] showed that a projector can not only be used for intraoperative visualization, but also for the registration of the patient's organ using structured light techniques. However, in preliminary studies in collaboration with the Institute of Process Control and Robotics, University of Karlsruhe, Germany [15] we found that complex 3D planning models (such as vascular structures) shaded with classical rendering methods are inappropriate for intraoperative projection.…”

Section: Augmented Reality For Liver Interventionsmentioning

confidence: 99%

See 1 more Smart Citation

Illustrative visualization of 3D planning models for augmented reality in liver surgery

et al. 2009

View full text Add to dashboard Cite

Purpose: Augmented Reality (AR) obtains increasing acceptance in the operating room. However, a meaningful augmentation of the surgical view with a 3D visualization of planning data which allows reliable comparisons of distances and spatial relations is still an open request. Methods: We introduce methods for intraoperative visualization of 3D planning models which extend illustrative rendering and AR techniques. We aim to reduce visual complexity of 3D planning models and accentuate spatial relations between relevant objects. The main contribution of our work is an advanced silhouette algorithm for 3D planning models (distance-encoding silhouettes) combined with procedural textures (distance-encoding surfaces). In addition, we present a method for illustrative visualization of resection surfaces. Results: The developed algorithms have been embedded into a clinical prototype that has been evaluated in the operating room. To verify the expressiveness of our illustration methods, we performed a user study under controlled conditions. The study revealed a clear advantage in distance assessment with the proposed illustrative approach in comparison to classical rendering techniques. Conclusion:The presented illustration methods are beneficial for distance assessment in surgical AR. To increase the safety of interventions with the proposed approach, reduction of inaccuracies in tracking and registration are subject of our current research.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Augmented Reality For Liver Interventionsmentioning

confidence: 99%

Illustrative visualization of 3D planning models for augmented reality in liver surgery

et al. 2009

View full text Add to dashboard Cite

show abstract

“…The Beamatron [15], a steerable AR system, can project graphics onto the surfaces of moving objects in real-time by using a Microsoft Kinect [16] as a depth sensor. A projector-based augmented-reality system [17] has been reported to offer an intuitive real-time intraoperative orientation in interstitial therapy. In order to reduce the delay time in projection mapping systems for dynamic scenes, a time-delay compensation algorithm [18] and a screen object-tracking algorithm [19] have been presented.…”

Section: Related Workmentioning

confidence: 99%

Real-Time Projection Mapping Using High-Frame-Rate Structured Light 3D Vision

Yamamoto

Aoyama

Takaki

et al. 2015

SICE Journal of Control, Measurement, and System Integration

View full text Add to dashboard Cite

: In this paper, the authors report on the development of a projection-mapping system that can project RGB light patterns that are enhanced for three dimensional (3D) scenes using a graphics processing unit (GPU) based highframe-rate (HFR) vision system synchronized with HFR projectors. The proposed system can acquire 512 × 512 depthimages in real time at 500 fps. The depth-images processing is accelerated by installing a GPU board for parallel processing of Gray-code structured light illumination using infrared (IR) light patterns projected from an IR projector. Using the computed depth-image, suitable RGB light patterns to be projected are generated in real time for enhanced application tasks. They are projected from an RGB projector as augmented information onto a 3D scene with pixel-wise correspondence even when the 3D scene is time-varied. Experimental results obtained from enhanced application tasks for time-varying 3D scenes such as (1) depth-based color mapping, (2) augmented reality (AR) spirit level and (3) AR wristwatch confirm the efficacy of our system.

show abstract

“…For example, headsets, hand-held devices and projector systems are all commonly employed in AR systems. The ability to integrate virtual objects into a real scene is invaluable in the medical industry [14][15][16][17] . Adding three-dimensional scan data to a view of a patient can provide visual support for both diagnosis and treatment.…”

Section: Introductionmentioning

confidence: 99%

A method for patient set-up guidance in radiotherapy using augmented reality

Talbot

Meyer

Watts

et al. 2009

Australas. Phys. Eng. Sci. Med.

View full text Add to dashboard Cite

A system for patient set-up in external beam radiotherapy was developed using Augmented Reality (AR). Live images of the linac treatment couch and patient were obtained with video cameras and displayed on a nearby monitor. A 3D model of the patient's external contour was obtained from planning CT data, and AR tracking software was used to superimpose the model onto the video images in the correct position for treatment. Throughout set-up and treatment, the user can view the monitor and visually confirm that the patient is positioned correctly. To ensure that the virtual contour was displayed in the correct position, a process was devised to register the coordinates of the linac with the camera images. A cube with AR tracking markers attached to its faces was constructed for alignment with the isocentre using room lasers or cone-beam CT. The performance of the system was investigated in a clinical environment by using it to position an anthropomorphic phantom without the aid of additional set-up methods. The positioning errors were determined by means of CBCT and image registration. The translational set-up errors were found to be less than 2.4 mm and the rotational errors less than 0.3 degrees. This proof-of-principle study has demonstrated the feasibility of using AR for patient position and pose guidance.

show abstract

Projector-Based Augmented Reality for Intuitive Intraoperative Guidance in Image-Guided 3D Interstitial Brachytherapy

Cited by 49 publications

References 28 publications

Illustrative visualization of 3D planning models for augmented reality in liver surgery

Illustrative visualization of 3D planning models for augmented reality in liver surgery

Real-Time Projection Mapping Using High-Frame-Rate Structured Light 3D Vision

A method for patient set-up guidance in radiotherapy using augmented reality

Contact Info

Product

Resources

About