ConfocalVR: Immersive Visualization for Confocal Microscopy

Abstract

ConfocalVR is a virtual reality (VR) application created to improve the ability of researchers to study the complexity of cell architecture. Confocal microscopes take pictures of fluorescently labeled proteins or molecules at different focal planes to create a stack of two-dimensional images throughout the specimen. Current software applications reconstruct the three-dimensional (3D) image and render it as a two-dimensional projection onto a computer screen where users need to rotate the image to expose the full 3D structure. This process is mentally taxing, breaks down if you stop the rotation, and does not take advantage of the eye's full field of view. ConfocalVR exploits consumer-grade VR systems to fully immerse the user in the 3D cellular image. In this virtual environment, the user can (1) adjust image viewing parameters without leaving the virtual space, (2) reach out and grab the image to quickly rotate and scale the image to focus on key features, and (3) interact with other users in a shared virtual space enabling real-time collaborative exploration and discussion. We found that immersive VR technology allows the user to rapidly understand cellular architecture and protein or molecule distribution. We note that it is impossible to understand the value of immersive visualization without experiencing it first hand, so we encourage readers to get access to a VR system, download this software, and evaluate it for yourself. The ConfocalVR software is available for download at http://www.confocalvr.com, and is free for nonprofits.

Virtual Reality and Oceanography: Overview, Applications, and Perspective

“…We imagine multiple users having the ability to meet in VR, and this could facilitate more meaningful discussion and analysis with participants viewing minimally curated, minimally biased data. For example, the application ConfocalVR 16 allows multiple users to interact in a virtual space and this has shown to be beneficial for understanding cellular structure (Stefani et al, 2018). VR-enabled collaboration could also be used for policy making, as has been previously done for coastal management and planning when stakeholders used these visualizations to assess the potential outcome of marine conservation and sustainability projects (Newell et al, 2017).…”

Bioscience education 2030 and beyond: Where will technology take the curriculum?

“…This allows the user to interact with the images, adjust parameters, and engage with other users in the same space. 22 In terms of molecular structures, one of the more powerful tools is UCSF ChimeraX, a next-generation molecular visualization tool that now supports VR. 23,24 In the last year, Nanome has launched a very powerful, collaborative, and user-friendly application for exploring molecular structures in a manner that is truly revolutionizing the way in which protein structure can be taught and researched.…”

Potential applications for virtual and augmented reality technologies in sensory science

“…The application of VR / AR as visualisation tools is gaining particular attention within the healthcare sector, and is already showing demonstrable results in facilitating medical interventions and enhancing the area of education and training (Aukstakalnis, 2017;Sutherland & La Russa, 2017). For instance, novel VR applications such as VRVisu, ConfocalVR, Arivis InViewR and Echopixel True 3D have the capability to reconstruct complex images from real-world medical datasets into real 3D immersive experiences, enabling medical practitioners to treat patients more effectively (Silva et al, 2018;Stefani, Lacy-Hulbert & Skillman, 2018;Reddivari, Smith & Pabalate, 2017). Augmented reality has also been recently adopted in medical practice whereby one of the UK's largest children's hospitals is exploring Microsoft Hololens for interactive visualization of 3D images of patient heart or brain scans during surgery (Palladino, 2018).…”