A 3-D Mixed-Reality System for Stereoscopic Visualization of Medical Dataset

Abstract

We developed a simple, light, and cheap 3-D visualization device based on mixed reality that can be used by physicians to see preoperative radiological exams in a natural way. The system allows the user to see stereoscopic "augmented images," which are created by mixing 3-D virtual models of anatomies obtained by processing preoperative volumetric radiological images (computed tomography or MRI) with real patient live images, grabbed by means of cameras. The interface of the system consists of a head-mounted display equipped with two high-definition cameras. Cameras are mounted in correspondence of the user's eyes and allow one to grab live images of the patient with the same point of view of the user. The system does not use any external tracker to detect movements of the user or the patient. The movements of the user's head and the alignment of virtual patient with the real one are done using machine vision methods applied on pairs of live images. Experimental results, concerning frame rate and alignment precision between virtual and real patient, demonstrate that machine vision methods used for localization are appropriate for the specific application and that systems based on stereoscopic mixed reality are feasible and can be proficiently adopted in clinical practice.

From CT scanning to 3-D printing technology for the preoperative planning in laparoscopic splenectomy

“…Nonetheless, full understanding of this information after simple scrolling of 2-D MDCT image slices is very difficult, even for the experienced surgeon. To overcome this problem, 3-D virtual reconstructions have been used to maximize understanding of the anatomy and increase the accuracy of the surgeon [9]. However, radiologists and surgeons are generally restricted by the use of flat screens for the visualization of 3-D images.…”

A Kinect-based Medical Augmented Reality System for Craniofacial Applications Using Image-to-Patient Registration

“…Recently, applying augmented reality (AR) technology to the IGNS is a trend for imageguided surgery. The medical AR system merges medical images or anatomical graphic models into the scene of real world [2][3][4]. Therefore, the ways to accurately align pre-operative medical images with physical anatomy and to effectively provide anatomical information to the surgeons are two important keys for amodern medical AR IGNS.…”

Robust and Accurate Algorithm for Wearable Stereoscopic Augmented Reality with Three Indistinguishable Markers

“…because they minimize the extra mental effort required to switch focus between the real surgical task and the augmented view presented on the external display. Wearable AR systems based on head-mounted displays (HMDs) intrinsically provide the user with an egocentric viewpoint and do not limit freedom of movement around the patient [22][23][24]. Standard HMDs provide both binocular parallax and motion parallax and smoothly augment the user's perception of the surgical scene throughout the specific surgical procedure.…”

“…In a previous work, we presented an early system based on a commercially available HMD equipped with two external cameras aligned to the user's eyes [23]. The see-through ability was created by combining 3D computer-generated models obtained by processing radiological images (e.g., CT or MRI) [30] with live views of the real patient.…”