Teleproctoring for Neurovascular Procedures: Demonstration of Concept Using Optical See-Through Head-Mounted Display, Interactive Mixed Reality, and Virtual Space Sharing—A Critical Need Highlighted by the COVID-19 Pandemic

Abstract

BACKGROUND AND PURPOSE: Physician training and onsite proctoring are critical for safely introducing new biomedical devices, a process that has been disrupted by the pandemic. A teleproctoring concept using optical see-through head-mounted displays with a proctor's ability to see and, more important, virtually interact in the operator's visual field is presented.MATERIALS AND METHODS: Test conditions were created for simulated proctoring using a bifurcation aneurysm flow model for WEB device deployment. The operator in the angiography suite wore a Magic Leap-1 optical see-through head-mounted display to livestream his or her FOV to a proctor's computer in an adjacent building. A Web-based application (Spatial) was used for the proctor to virtually interact in the operator's visual space. Tested elements included the quality of the livestream, communication, and the proctor's ability to interact in the operator's environment using mixed reality. A hotspot and a Wi-Fi-based network were tested. RESULTS:The operator successfully livestreamed the angiography room environment and his FOV of the monitor to the remotely located proctor. The proctor communicated and guided the operator through the procedure over the optical see-through head-mounted displays, a process that was repeated several times. The proctor used mixed reality and virtual space sharing to successfully project images, annotations, and data in the operator's FOV for highlighting any device or procedural aspects. The livestream latency was 0.71 (SD, 0.03) seconds for Wi-Fi and 0.86 (SD, 0.3) seconds for the hotspot (P ¼ .02). The livestream quality was subjectively better over the Wi-Fi.CONCLUSIONS: New technologies using head-mounted displays and virtual space sharing could offer solutions applicable to remote proctoring in the neurointerventional space.

Mixed Reality in Modern Surgical and Interventional Practice: Narrative Review of the Literature

“…Moreover, it allows for telementoring through data sharing and communication with other surgical team members or surgical residents [ 45 ]. Telementoring by off-site experts using HMD has already been demonstrated in neurovascular procedures and cancer surgery [ 58 , 59 ].…”

Augmenting Performance: A Systematic Review of Optical See-Through Head-Mounted Displays in Surgery

“…Robot-assisted surgery applications included orthopedic applications [ 55 ], general bedside tasks [ 56 ], and endoluminal interventions [ 57 ]. Papers focused on general surgery and associated applications included telementoring [ 58 ], teleproctoring [ 59 ], surgical navigation [ 60 ], and liver resection [ 61 ]. Neurosurgery-related applications included incision planning [ 62 , 63 ], navigation [ 64 , 65 ], and complex craniotomies [ 66 ].…”

“…Ultrasound imaging was the second most frequently used form of intraoperative data, and is commonly used as a tool for confirming real-time tissue targets in needle biopsy procedures [ 82 , 83 ]. Telestrations, such as virtual arrows and other virtual annotations were common in telementoring and teleproctoring [ 58 , 59 , 60 ]. Intraoperative endoscope video is the primary real-time data source for laparoscopic and endoscopic procedures and robot-assisted surgery [ 56 , 82 ].…”

“…Not all virtually augmented content needs to be inherently 3D for it to provide value during surgical navigation. Several groups reported the use of 2D information in the form of telestrations for telementoring in general surgical task training [ 58 ] or teleproctoring for assistance during neurovascular procedures [ 59 ]. Other uses of 2D data included the display of raw 2D ultrasound data for general intraprocedural guidance [ 83 , 92 ] and assistance during cervical pedicle screw placement [ 93 ].…”

The Advantage of Using an Optical See-Through Head-Mounted Display in Ultrasonography-Guided Needle Biopsy Procedures: A Prospective Randomized Study

“…An OST-HMD enables practitioners to see the images on a monitor just in front of their eyes with their hands free. Its advantages have been reported in central venous catheterization, spinal puncture, and orthopedic and neurosurgery [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ]. Thus, when the OST-HMD is applied to US-guided needle biopsy procedures, the practitioner can perform the insertion of the needle without moving their heads or bodies, which is expected to make the puncture procedure more accurate and safer.…”