Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Velazco-Garcia, Jose D.; Navkar, Nikhil V.; Balakrishnan, Shidin; Younes, Georges; Abinahed, Julien; Al-Rumaihi, Khalid; Darweesh, Adham; Elakkad, Mohamed S; Al‐Ansari, Abdulla; Christoforou, Eftychios G.; Karkoub, Mansour; Leiss, Ernst L.; Tsiamyrtzis, Panagiotis; Tsekos, Nikolaos V.

doi:10.1002/rcs.2290

Cited by 27 publications

(12 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These programs can also often aid the performance of the actual surgical procedure itself, and can overlay patient-specific anatomical information obtained from imaging scans, such as in spinal surgeries [ 30 , 31 ]. Transrectal prostate biopsy practice and training can also be aided by this technology, to help control robot-assisted apparatus [ 32 ]. Multiple AR-based programs are already used for kidney surgery and training (and several more are likely to be soon adopted), including those to aid patient education [ 33 , 34 ].…”

Section: Resultsmentioning

confidence: 99%

Augmented reality in medical education: students’ experiences and learning outcomes

Dhar

Rocks

Samarasinghe

et al. 2021

Medical Education Online

139

View full text Add to dashboard Cite

Augmented reality (AR) is a relatively new technology that allows for digitally generated three-dimensional representations to be integrated with real environmental stimuli. AR can make use of smart phones, tablets, or other devices to achieve a highly stimulating learning environment and hands-on immersive experience. The use of AR in industry is becoming widespread with applications being developed for use not just for entertainment and gaming but also healthcare, retail and marketing, education, military, travel and tourism, automotive industry, manufacturing, architecture, and engineering. Due to the distinct learning advantages that AR offers, such as remote learning and interactive simulations, AR-based teaching programs are also increasingly being adopted within medical schools across the world. These advantages are further highlighted by the current COVID-19 pandemic, which has caused an even greater shift towards online learning. In this review, we investigate the use of AR in medical training/education and its effect on students' experiences and learning outcomes. This includes the main goals of AR-based learning, such as to simplify the delivery and enhance the comprehension of complex information. We also describe how AR can enhance the experiences of medical students, by improving knowledge and understanding, practical skills and social skills. These concepts are discussed within the context of specific AR medical training programs, such as HoloHuman, OculAR SIM, and HoloPatient. Finally, we discuss the challenges of AR in learning and teaching and propose future directions for the use of this technology in medical education.

show abstract

Section: Resultsmentioning

confidence: 99%

Augmented reality in medical education: students’ experiences and learning outcomes

Dhar

Rocks

Samarasinghe

et al. 2021

Medical Education Online

139

View full text Add to dashboard Cite

show abstract

“…The majority of papers made use of preoperative CT data as the input to the surgical guidance strategy, corresponding with the frequent application space of orthopedic surgery and the enhanced ability of CT to generate geometrically accurate representations of rigid anatomical structures, such as bone [ 45 , 76 , 77 ]. The second most frequently used data type was MRI, either as a standalone data type [ 62 , 72 , 78 ] or in combination with CT [ 49 , 52 , 53 ]. Magnetic resonance imaging has better sensitivity than CT when imaging soft tissue structures; however, it is less geometrically precise [ 79 ] and has less signal intensity when imaging bony structures.…”

Section: Resultsmentioning

confidence: 99%

“…During an OST-HMD led surgical navigation procedure, if the surgeon (or a bystander) is required to manually adapt or manipulate the presented virtual content to be relevant to their current surgical context it can introduce additional ease of use and interaction challenges. Several articles mentioned workflow challenges due to the manual interaction with virtual models required to adapt the information to the current surgical task [ 33 , 69 , 78 , 83 ]. These workflow challenges contributed to reduced enjoyment and overall usability of the guidance system [ 58 , 86 ].…”

Section: Resultsmentioning

confidence: 99%

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

2022

View full text Add to dashboard Cite

We conducted a systematic review of recent literature to understand the current challenges in the use of optical see-through head-mounted displays (OST-HMDs) for augmented reality (AR) assisted surgery. Using Google Scholar, 57 relevant articles from 1 January 2021 through 18 March 2022 were identified. Selected articles were then categorized based on a taxonomy that described the required components of an effective AR-based navigation system: data, processing, overlay, view, and validation. Our findings indicated a focus on orthopedic (n=20) and maxillofacial surgeries (n=8). For preoperative input data, computed tomography (CT) (n=34), and surface rendered models (n=39) were most commonly used to represent image information. Virtual content was commonly directly superimposed with the target site (n=47); this was achieved by surface tracking of fiducials (n=30), external tracking (n=16), or manual placement (n=11). Microsoft HoloLens devices (n=24 in 2021, n=7 in 2022) were the most frequently used OST-HMDs; gestures and/or voice (n=32) served as the preferred interaction paradigm. Though promising system accuracy in the order of 2–5 mm has been demonstrated in phantom models, several human factors and technical challenges—perception, ease of use, context, interaction, and occlusion—remain to be addressed prior to widespread adoption of OST-HMD led surgical navigation.

show abstract

“…This information, pertaining to open surgery operative field, can be rendered in an immersive environment on a virtual reality display for the mentor [38]. On the other hand, a head-mounted display can be used to render dynamic holograms of virtual surgical instruments motion onto the view of the mentee [39][40][41]. Second, there is potential for the software modules of the proposed tele-mentoring system to be integrated with existing commercial platforms [5,31,42].…”

Section: Discussionmentioning

confidence: 99%

Preliminary design and evaluation of a remote tele-mentoring system for minimally invasive surgery

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background Tele-mentoring during surgery facilitates the transfer of surgical knowledge from a mentor (specialist surgeon) to a mentee (operating surgeon). The aim of this work is to develop a tele-mentoring system tailored for minimally invasive surgery (MIS) where the mentor can remotely demonstrate to the mentee the required motion of the surgical instruments. Methods A remote tele-mentoring system is implemented that generates visual cues in the form of virtual surgical instrument motion overlaid onto the live view of the operative field. The technical performance of the system is evaluated in a simulated environment, where the operating room and the central location of the mentor were physically located in different countries and connected over the internet. In addition, a user study was performed to assess the system as a mentoring tool. Results On average, it took 260 ms to send a view of the operative field of 1920 × 1080 resolution from the operating room to the central location of the mentor and an average of 132 ms to receive the motion of virtual surgical instruments from the central location to the operating room. The user study showed that it is feasible for the mentor to demonstrate and for the mentee to understand and replicate the motion of surgical instruments. Conclusion The work demonstrates the feasibility of transferring information over the internet from a mentor to a mentee in the form of virtual surgical instruments. Their motion is overlaid onto the live view of the operative field enabling real-time interactions between both the surgeons.

show abstract

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Cited by 27 publications

References 61 publications

Augmented reality in medical education: students’ experiences and learning outcomes

Augmented reality in medical education: students’ experiences and learning outcomes

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

Preliminary design and evaluation of a remote tele-mentoring system for minimally invasive surgery

Contact Info

Product

Resources

About