Comparison of optical see-through head-mounted displays for surgical interventions with object-anchored 2D-display

Abstract

Purpose Optical see-through head-mounted displays (OST-HMD) feature an unhindered and instantaneous view of the surgery site and can enable a mixed reality experience for surgeons during procedures. In this paper, we present a systematic approach to identify the criteria for evaluation of OST-HMD technologies for specific clinical scenarios, which benefit from using an object-anchored 2D-display visualizing medical information. Methods Criteria for evaluating the performance of OST-HMDs for visualization of medical information and its usage are identified and proposed. These include text readability, contrast perception, task load, frame rate, and system lag. We choose to compare three commercially available OST-HMDs, which are representatives of currently available head-mounted display technologies. A multi-user study and an offline experiment are conducted to evaluate their performance. Results Statistical analysis demonstrates that Microsoft HoloLens performs best among the three tested OST-HMDs, in terms of contrast perception, task load, and frame rate, while ODG R-7 offers similar text readability. The integration of indoor localization and fiducial tracking on the HoloLens provides significantly less system lag in a relatively motionless scenario. Conclusions With ever more OST-HMDs appearing on the market, the proposed criteria could be used in the evaluation of their suitability for mixed reality surgical intervention. Currently, Microsoft HoloLens may be more suitable than ODG R-7 and Epson Moverio BT-200 for clinical usability in terms of the evaluated criteria. To the best of our knowledge, this is the first paper that presents a methodology and conducts experiments to evaluate and compare OST-HMDs for their use as object-anchored 2D-display during interventions.

Virtual, augmented, and mixed reality applications in orthopedic surgery

“…Although a growing number of systems have been introduced to the market, Microsoft has forged the path in the mixed reality domain with the HoloLens headset device, enabling the user to control the headset by verbal command and hand gesture. A recent literature study on the evaluation of OST‐HMD suitability for mixed‐reality surgical intervention demonstrated that Microsoft HoloLens outperforms other currently available OST HMDs …”

Exploring mixed reality based on self-efficacy and motivation of users

“…Those technologies include spatial AR (in which displays are fixed and users need not wear or hold a display device), video see-through and optical see-through. Effectively a portable computer such as the Microsoft HoloLens is an optical see-through HMD, providing for the reflection of projected images without significant visual obstruction of the real world (Qian et al 2017). HoloLens users experience a stable perception of the overlaid image and can either move the hologram or move themselves freely around the hologram, strengthening the users' perception of the 3D environment (Bach et al 2018;van der Meulen et al 2017).…”

“…Despite privacy concerns, one first step study found that patients were open to improved patient care through doctors' use of HMD (Prochaska et al 2016). Psychological issues such as phobias may be overcome through the use of VR exposure therapy (Coppens 2017), and the optical see-through display may lend itself to use in surgery (Qian et al 2017).…”

Interventional 3D Augmented Reality in Orthopedic, Trauma and Vascular Surgery

“…With the advent of modern mixed-reality head-mounted displays (HMDs), the typical challenges encountered by mobile AR technologies have now been solved, and commercial devices suitable for medical applications are now available. Indeed, a recent study showed that the Microsoft's HoloLens HMDs are now suitable enough in terms of contrast perception, task load and frame rate, during surgical interventions (23). Thus, the HMD not only provides an accurate feedback about the current radiation exposure to surgeon and patient, but also correlates such an exposure to the underlying workflow tasks taking place.…”