MIXR: A Standard Architecture for Medical Image Analysis in Augmented and Mixed Reality

Abstract

Medical image analysis is evolving into a new dimension: where it will combine the power of AI and machine learning with real-time, real-space displays, namely Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) -known collectively as Extended Reality (XR). These devices, typically available as head-mounted displays, are enabling the move towards the complete transformation of how medical data is viewed, processed and analysed in clinical practice. There have been recent attempts on how XR gadgets can help in surgical planning and training of medics. However, the radiological front from a detection, diagnostics and prognosis remains unexplored.In this paper we propose a standard framework or architecture called Medical Imaging in Extended Reality (MIXR) for building medical image analysis applications in XR. MIXR consists of several components used in literature; however, tied together for reconstructing volume data in 3D space. Our focus here is on the reconstruction mechanism for CT and MRI data in XR; nevertheless, the framework we propose has applications beyond these modalities.

Beyond Your Sight Using Metaverse Immersive Vision With Technology Behaviour Model

“…Operational value refers to the value of surgery performed with ex-tended reality technology. As seen in the actual operational use by surgeons, XR technology can be simulated and practised before surgery to anticipate the risks that exist during surgery and facilitate the performance of difficult surgeries (Allison et al, 2020;Wilfredo López-Ojeda, 2022). Intraoperative use of interaction allows for remote communication, random image finding, improved human eye vision, and supervision of the surgical procedure by the surgical specialist.…”

Augmented Reality in Surgery: A Scoping Review

“…Its origin can be dated back to 1895 when W. C. Roentgen discovered the existence of X-rays [2]. While in the course of the twentieth century, X-rays have found increasing application, in more recent years, other techniques have been developed and acquiring data from the internal structures of the human body has become more and more useful [1,[3][4][5]. All this facilitated an increasing use of images to guide surgeons during interventions, leading to the affirmation of image-guided surgery (IGS) [6].…”

“…All this facilitated an increasing use of images to guide surgeons during interventions, leading to the affirmation of image-guided surgery (IGS) [6]. In this sense, the need for reducing surgery evasiveness, by supporting physicians in the diagnosis and preoperative phases as well as during surgeries themselves, led to the use of different solutions such as the 3D visualization of anatomical parts and the application of augmented reality (AR) in surgery [1,3,4]. Augmented reality consists in merging the real word with virtual objects (VOs) generated by computer graphic systems, creating a world for the user that is augmented with VOs.…”