A modular and scalable computational framework for interactive immersion into imaging data with a holographic augmented reality interface

Velazco-Garcia, Jose D.; Shah, Dipan J.; Leiss, Ernst L.; Tsekos, Nikolaos V.

doi:10.1016/j.cmpb.2020.105779

Cited by 10 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The application was developed in C++ as a module built on top of the Framework for Interactive Immersion into Imaging Data. 35 The framework enabled the described system with the AR visualisation capabilities and communication protocol for the two entities to exchange visual data and user input. The framework allowed us to focus in the development of the studies without needing to manage the technical details of the application, that is, rendering of images and F I G U R E 1 (A) Schematic representation of the study setup illustrating the different components and their interaction: (i) The interventional planning workstation receives MR images from the scanner and computes the actuation commands to manipulate a virtual robotic manipulator for placement of the probe.…”

Section: Methodsmentioning

confidence: 99%

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Velazco-Garcia

Navkar

Balakrishnan

et al. 2021

Robotics Computer Surgery

Self Cite

View full text Add to dashboard Cite

Background: User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment. Method:End-user studies were conducted by simulating an MRgPBx system with end-and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system. Results:The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system. Conclusions:The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning.

show abstract

Section: Methodsmentioning

confidence: 99%

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Velazco-Garcia

Navkar

Balakrishnan

et al. 2021

Robotics Computer Surgery

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although AR has great potential, the hardware is not yet user-friendly [8]. Recent work introduced the use of AR as a tool for processing and visualization of imaging data which can be subtracted from medical devices such as MRI scanners, simulation of surgical tools, and other assistive data [9]. This integration of the physician with the data and sensors ensures the visualization of patient data in 3D and collects and analyses newly generated data.…”

Section: Ar Application Fieldsmentioning

confidence: 99%

How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective

et al. 2021

View full text Add to dashboard Cite

In the last few decades, several initiatives and approaches are set up to support maintenance procedures for the railway industry in adopting the principles of Industry 4.0. Contextualized maintenance technologies such as Augmented Reality (AR) overlay can integrate virtual information on physical objects to improve decision-making and action-taking processes. Operators work in a dynamic working environment requiring both high adaptive capabilities and expert knowledge. There is a need to support the operators with tailor-based information that is customized and contextualized to their expertise and experience. It calls for AR tools and approaches that combine complex methodologies with high usability requirements. The development of these AR tools could benefit from a structured approach. Therefore, the objective of this paper is to propose an adaptive architectural framework aimed at shaping and structuring the process that provides operators with tailored support when using an AR tool. Case study research is applied within a revelatory railway industry setting. It was found that the framework ensures that self-explanatory AR systems can capture the knowledge of the operator, support the operator during maintenance activities, conduct failure analysis, provide problem-solving strategies, and improve learning capabilities. This study contributes to the necessity of having a human-centered approach for the successful adaption of AR technology tools for the railway industry.

show abstract

“…One of the practical challenges and an important requirement in deploying modern human-immersion systems is providing computational power for on-the-fly processing and analytics [15,[28][29][30]. Holographic immersion systems perform a considerable number of calculations [17] to process and reconstruct a 3D mesh and hologram on holographic displays, which requires processing power [15,31,32]. Also, with the increasing integration of AI-driven models for real-time data analysis and diagnosis, providing more computational resources is vital to avoid data delivery delays [29,30,33,34].…”

Section: Introductionmentioning

confidence: 99%

“…Providing such an environment requires high processing power, data exchange, and memory to retain the information of the holograms, which seems challenging to fulfill by the current generation of XR limited hardware capabilities. Therefore, to address this issue, system designs [15,17,31,[35][36][37] rely on external servers or processing units to perform these crucial functionalities to reduce resource utilization on the HMD or HHD.…”

Section: Introductionmentioning

confidence: 99%

“…As such, it can be used in medical training and diagnosis using virtually any medical image modality as input, such as X-rays, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, positron emission tomography (PET), etc. Of particular value, however, is in interventional medicine when used with HMD to support hands-free interventions [31,42].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Multiuser, Multisite, and Platform-Independent On-the-Cloud Framework for Interactive Immersion in Holographic XR

Neeli,

Tran,

Velazco-Garcia

et al. 2024

Applied Sciences

Self Cite

View full text Add to dashboard Cite

Background: The ever-growing extended reality (XR) technologies offer unique tools for the interactive visualization of images with a direct impact on many fields, from bioinformatics to medicine, as well as education and training. However, the accelerated integration of artificial intelligence (AI) into XR applications poses substantial computational processing demands. Additionally, the intricate technical challenges associated with multilocation and multiuser interactions limit the usability and expansion of XR applications. Methods: A cloud deployable framework (Holo-Cloud) as a virtual server on a public cloud platform was designed and tested. The Holo-Cloud hosts FI3D, an augmented reality (AR) platform that renders and visualizes medical 3D imaging data, e.g., MRI images, on AR head-mounted displays and handheld devices. Holo-Cloud aims to overcome challenges by providing on-demand computational resources for location-independent, synergetic, and interactive human-to-image data immersion. Results: We demonstrated that Holo-Cloud is easy to implement, platform-independent, reliable, and secure. Owing to its scalability, Holo-Cloud can immediately adapt to computational needs, delivering adequate processing power for the hosted AR platforms. Conclusion: Holo-Cloud shows the potential to become a standard platform to facilitate the application of interactive XR in medical diagnosis, bioinformatics, and training by providing a robust platform for XR applications.

show abstract

A modular and scalable computational framework for interactive immersion into imaging data with a holographic augmented reality interface

Cited by 10 publications

References 16 publications

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

How to Make Augmented Reality a Tool for Railway Maintenance Operations: Operator 4.0 Perspective

A Multiuser, Multisite, and Platform-Independent On-the-Cloud Framework for Interactive Immersion in Holographic XR

Contact Info

Product

Resources

About