Augmented Reality and Virtual Reality: Initial Successes in Diagnostic Radiology

Abstract

In this chapter, we will review the applications of augmented reality (AR) and virtual reality (VR) in diagnostic radiology. We begin the chapter by discussing state of the art medical imaging techniques to include scanner hardware, spatial resolution, and presentation methods to the radiologist or other medical professionals. We continue by discussing the methodology of a technique called Depth-3-Dimensional (D3D) imaging, which transforms cross-sectional medical imaging datasets into a left and right eye images that that can be viewed with state of the art AR or VR headsets. We include results of the D3D processed AR/VR imaging and conclude with a discussion of the path forward.

Visual Search Performance and Strategy for Three-Dimensional Visualization Systems: Impact of Radiologist Experience

“…Altogether, the short search times and minimal amount of image manipulations demonstrated by experienced radiologists suggest that three-dimensional visualization may potentially become a helpful tool in the reading of medical images in the future. Three-dimensional visualization methods such as VR and AR are intended to improve ergonomics in radiology [14][15][16][17][18][19][20] by reducing the cognitive load and time required to inspect multi-slice images. In contrast, viewing the entire three-dimensional image on the display could be overwhelming to students and early-career radiologists, due to the dominance of bottom-up processing in visual search.…”

“…The rendering of two-dimensional images is discussed as a possible solution to this issue [1,13]. Namely, cross-sectional images can be fused into a three-dimensional representation of data, for instance, in virtual reality (VR) [14,15], augmented reality (AR) [15,16], on stereoscopic displays [17,18], or on volumetric displays [19,20].…”

“…In the human-centered design of a three-dimensional visualization system, one should consider the alternation of global and local search when radiologists read medical images [41], and ensure flexible and meaningful navigation through the rendered images, which will facilitate both modes of search. As an alternative to traditionally used navigation devices, a gaming controller has been proposed to facilitate visual search in the three-dimensional environment [1,16]. The practicality of a new navigation device has not yet been reported and should be assessed using quantitative and qualitative measures.…”

Breast3D: An Augmented Reality System for Breast CT and MRI

“…Most of the work in the area in general is within surgical planning, nevertheless, specifically for breast cancer using MRI data. However, there are even fewer examples of XR in breast cancer radiology [5].…”

“…XR technology has yet to reach its full potential for radiology, preliminary systems have started to be proposed for breast cancer diagnosis. The system D3D [5]- [7] is the only known XR system for breast cancer diagnosis. D3D showed promising results demonstrating the depth perception and focal point convergence of XR systems in medical imaging.…”