Enabling Real-Time Volume Rendering of Functional Magnetic Resonance Imaging on an iOS Device

Abstract

Powerful non-invasive imaging technologies like computed tomography (CT), ultrasound, and magnetic resonance imaging (MRI) are used daily by medical professionals to diagnose and treat patients. While 2D slice viewers have long been the standard, many tools allowing 3D representations of digital medical data are now available. The newest imaging advancement, functional MRI (fMRI) technology, has changed medical imaging from viewing static to dynamic physiology (4D) over time, particularly to study brain activity. Add this to the rapid adoption of mobile devices for everyday work and the need to visualize fMRI data on tablets or smartphones arises. However, there are few mobile tools available to visualize 3D MRI data, let alone 4D fMRI data. Building volume rendering tools on mobile devices to visualize 3D and 4D medical data is challenging given the limited computational power of the devices. This paper describes research that explored the feasibility of performing real-time 3D and 4D volume raycasting on a tablet device. The prototype application was tested on a 9.7" iPad Pro using two different fMRI datasets of brain activity. The results show that mobile raycasting is able to achieve between 20 and 40 frames per second for traditional 3D datasets, depending on the sampling interval, and up to 9 frames per second for 4D data. While the prototype application did not always achieve true real-time interaction, these results clearly demonstrated that visualizing 3D and 4D digital medical data is feasible with a properly constructed software framework.

Effective volume rendering on mobile and standalone VR headsets by means of a hybrid method

“…Far from interactive visualization no more than 4-5 Frames Per Second (FPS), doubled with empty space skipping, could be reached in the case of a common medical volume visualization task (detailed in later sections). Recent studies [16], with no VR but mobile applications in scope, also confirm these results. In their paper of this year [3] Sutherland and his colleagues claim that rendering of most medical datasets on mobile VR platforms is 'likely to be too challenging'.…”

“…The reached FPS was fairly low (4.7 without and 8.5 with optimization) due to the long-running fragment shaders. Although no test results were found in the literature about the speed of ray casting visualization on mobile and standalone VR platforms, the measurements of Holub and Winer [16] confirmed the above values. They displayed the structural (static) component of a fMRI dataset with 256x256x128 resolution on an 2016 iPad Pro and got about 18 FPS.…”

Altered functional connectivity after pilocarpine-induced seizures revealed by intrinsic optical signals imaging in awake mice

“…In addition, IOS was performed with relatively non-invasive transparent craniotomy and head-fixing apparatus in awake mice and is easily amendable to integrate with EEG. 20,21 For this study, a pilocarpine (PILO) induced seizure model was chosen due to its widespread use in generating seizures in mice and its convenience to be applied through intraperitoneal injection. 22 In this study, we found hemoglobin content was changed after seizure induction and that it progressively, but only partly, returned to normal when seizures ended.…”

“…By capitalizing on rest-state activity, large-scale functional networks with the topographical organization were revealed using IOS. In addition, IOS was performed with relatively non-invasive transparent craniotomy and head-fixing apparatus in awake mice and is easily amendable to integrate with EEG 20 , 21 . For this study, a pilocarpine (PILO) induced seizure model was chosen due to its widespread use in generating seizures in mice and its convenience to be applied through intraperitoneal injection 22 …”

Region Space Guided Transfer Function Design for Nonlinear Neural Network Augmented Image Visualization

“…Hsieh et al [7] visualized the three-dimensional (3D) geometry of the ear ossicle with the segmented ossicle computer tomography (CT) slices, which presented the spatial relation with the temporal bone to diagnose middle ear disease. To visualize brain activity conveniently, Holub and Winer [8] performed 3D and 4D volume ray casting on a tablet device in real-time.…”