O49 The use of augmented reality for surgical guidance: an assessment of the human error

Abstract

Introduction Microsoft HoloLens® is an augmented-reality headset which is increasingly used for surgical guidance. This headset allows the overlay of patient-specific virtual models obtained from medical images onto the patient’s body surface using automatic marker-based alignment. This can guide surgeons during certain surgical tasks, e.g. determining biopsy needle entry points. This study aims to measure the human error in the localisation of virtual models with the headset and discuss its surgical implications. Method 59 adults were recruited between the ages of 20–59 years. A 12 cm2 digital marker was displayed on a monitor in 9 different positions, one at a time. This was repeated 3 times, resulting in 27 markers shown to each participant. Once a marker was detected by the headset’s camera, a virtual hexagon was rendered on the headset’s transparent lenses. Participants were tasked to click on the hexagon’s vertices using a mouse. The clicks’ coordinates were recorded by the system and compared to the predicted coordinates. This allowed for the calculation of the vertex localisation error. Result The mean vertex localisation error was found to be 5.19 mm (±3.56) with a range from 0.08 to 29.77 mm. There was a significant difference between marker positions as determined by a one-way ANOVA (P < 0.001). Conclusion This study suggests that the error in the localisation of virtual models depends on the position of the markers relative to the user wearing the headset. Further research is required to explore whether training can reduce the human error with this headset. Take-home Message The range of the human error in localising virtual models via the Microsoft HoloLens® headset is large and may be dependent on the position of the marker relative to the user of the headset. Further research is needed to investigate whether training with the headset can improve human performance.