The System for Telementoring with Augmented Reality (STAR): A head-mounted display to improve surgical coaching and confidence in remote areas

Rojas-Muñoz, Edgar; Cabrera, María Eugenia; Lin, Chengyuan; Andersen, Daniel; Popescu, Voicu; Anderson, Kathryn; Zarzaur, Ben L.; Mullis, Brian; Wachs, Juan P.

doi:10.1016/j.surg.2019.11.008

Cited by 53 publications

(67 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Participants were unexperienced surgeons (surgical residents and medical students) and were divided into two different groups: the former receiving remote instructions provided by an expert surgeon, directly on their field of view, using the STAR system; the latter receiving no external guidance beyond initial consultation of the Advanced Surgical Skills for Exposure in Trauma course manual. Results showed fewer mistakes and better performances among mentees belonging to the group that received guidance trough the STAR system (18).…”

Section: Discussionmentioning

confidence: 97%

Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

et al. 2021

View full text Add to dashboard Cite

Background: While performing surgeries in the OR, surgeons and assistants often need to access several information regarding surgical planning and/or procedures related to the surgery itself, or the accessory equipment to perform certain operations. The accessibility of this information often relies on the physical presence of technical and medical specialists in the OR, which is increasingly difficult due to the number of limitations imposed by the COVID emergency to avoid overcrowded environments or external personnel. Here, we analyze several scenarios where we equipped OR personnel with augmented reality (AR) glasses, allowing a remote specialist to guide OR operations through voice and ad-hoc visuals, superimposed to the field of view of the operator wearing them.Methods: This study is a preliminary case series of prospective collected data about the use of AR-assistance in spine surgery from January to July 2020. The technology has been used on a cohort of 12 patients affected by degenerative lumbar spine disease with lumbar sciatica co-morbidities. Surgeons and OR specialists were equipped with AR devices, customized with P2P videoconference commercial apps, or customized holographic apps. The devices were tested during surgeries for lumbar arthrodesis in a multicenter experience involving author's Institutions.Findings: A total number of 12 lumbar arthrodesis have been performed while using the described AR technology, with application spanning from telementoring (3), teaching (2), surgical planning superimposition and interaction with the hologram using a custom application for Microsoft hololens (1). Surgeons wearing the AR goggles reported a positive feedback as for the ergonomy, wearability and comfort during the procedure; being able to visualize a 3D reconstruction during surgery was perceived as a straightforward benefit, allowing to speed-up procedures, thus limiting post-operational complications. The possibility of remotely interacting with a specialist on the glasses was a potent added value during COVID emergency, due to limited access of non-resident personnel in the OR.Interpretation: By allowing surgeons to overlay digital medical content on actual surroundings, augmented reality surgery can be exploited easily in multiple scenarios by adapting commercially available or custom-made apps to several use cases. The possibility to observe directly the operatory theater through the eyes of the surgeon might be a game-changer, giving the chance to unexperienced surgeons to be virtually at the site of the operation, or allowing a remote experienced operator to guide wisely the unexperienced surgeon during a procedure.

show abstract

Section: Discussionmentioning

confidence: 97%

Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Second, equipped with a camera and video-recording capabilities, this new device could become a more useful adjunct tool in surgical education, information sharing, real-time consultation, and remote instruction and monitoring. 1,11,12 There are some concerns to be noted, such as the wearability of and fatigue caused by using a smart glasses device. We did not conduct a survey to assess feedback from multiple users on the usability and comfort of wearing such a device attached to glasses.…”

Section: Discussionmentioning

confidence: 99%

Smart glasses display device for fluoroscopically guided minimally invasive spinal instrumentation surgery: a preliminary study

Matsukawa¹,

Yato²

2021

Journal of Neurosurgery: Spine

View full text Add to dashboard Cite

OBJECTIVEMost surgeons are forced to turn their heads away from the surgical field to see various intraoperative support monitors. These movements may result in inconvenience to surgeons and lead to technical difficulties and potential errors. Wearable devices that can be attached to smart glasses or any glasses are novel visualization tools providing an alternative screen in front of the user’s eyes, allowing surgeons to keep their attention focused on the operative task without taking their eyes off the surgical field. The aim of the present study was to examine the feasibility of using glasses equipped with a wearable display device that transmits display monitor data during fluoroscopically guided minimally invasive spinal instrumentation surgery.METHODSIn this pilot prospective randomized study, 20 consecutively enrolled patients who underwent single-segment posterior lumbar interbody fusion (PLIF) at L5–S1 performed using the percutaneous pedicle screw technique were randomly divided into two groups, a group for which the surgeon used a wearable display device attached to regular glasses while performing surgery (smart glasses group) and a group for which the surgeon did not use such a device (nonglasses group). Real-time intraoperative fluoroscopic images were wirelessly transmitted to the display device attached to the surgeon’s glasses. The number of head turns performed by the surgeon to view the standard fluoroscopic monitor during procedures and the operative time, estimated blood loss, radiation exposure time, screw placement accuracy, and intraoperative complication rate were evaluated for comparison between the two groups.RESULTSThe number of surgeon head turns to view the fluoroscopic monitor in the smart glasses group was 0.10 ± 0.31 times, which was significantly fewer than the head turns in the nonglasses group (82.4 ± 32.5 times; p < 0.001). The operative and radiation exposure times in the smart glasses group were shorter than those in the nonglasses group (operative time 100.2 ± 10.4 vs 105.5 ± 14.6 minutes, radiation exposure time 38.6 ± 6.6 vs 41.8 ± 16.1 seconds, respectively), although the differences were not significant. Postoperative CT showed one screw perforation in the nonglasses group, and no intraoperative complications were observed in either group.CONCLUSIONSThis is, to the authors’ knowledge, the first report on the feasibility of using this wearable display device attached to glasses for fluoroscopically guided minimally invasive spinal instrumentation surgery. Smart glasses display devices such as this one may be a valid option to facilitate better concentration on operative tasks by improving ergonomic efficiency during surgery.

show abstract

“…Diese Daten können dann wiederum in chirurgische Lehrsoftware eingespielt werden und nützliche Informationen z. B. zur Handführung bei Präparation des Gewebes im Sinne eines Overlays auf dem Bildschirm zeigen [8,9]. Durch das Sammeln der chirurgischen Video-und Sensordaten weltweit könnte eine globale Datenbank erstellt werden, die die Rohdaten der Performance in Bezug zu klinischen Outcomes setzt und evidenzbasierte Verfahren für jeden individuellen Fall in Echtzeit vorschlägt [4].…”

Section: Ki Und Chirurgische Performanceunclassified

“…Nehmen wir als Beispiel die Doppelkonsolen-Prinzip des DaVinci-Roboters: Der Mentor und Mentee sitzen an 2 separaten Roboter-Konsolen -die räumliche Entfernung spielt dabei theoretisch keine Rolle -und operieren gemeinsam. Der Mentor hat die Möglichkeit, mit in das Bild projizierten Pfeilen, den jungen Chirurgen auf die wichtigen Strukturen hinzuweisen oder den geeigneten Präparationsweg anzuzei-Display, welches dem Chirurgen eine direkte Sicht auf den Situs durch einen Tablet-Computer ermöglicht[8]. Dabei handelt es sich um Systeme, die nicht nur im experimentellen Setting überprüft wurden, sondern vielmehr bereits im echten Leben zum Einsatz kommen -insbesondere in Kriegsgebieten und entlegenen Orten.…”

unclassified

Machine Learning and Learning Machines: KI in der Aus- und Weiterbildung

et al. 2020

View full text Add to dashboard Cite

ZUSAMMENFASSUNGDie Künstliche Intelligenz (KI) geht Hand in Hand mit der Digitalisierung der chirurgischen Aus- und Weiterbildung und wird in der Zukunft immer mehr an Bedeutung gewinnen. Die Anforderungen an KI in der chirurgischen Aus- und Weiterbildung sind vielfältig – genauso vielfältig sind ihre Einsatzmöglichkeiten. KI wird zunehmend im chirurgischen Curriculum berücksichtigt, auch wenn sie manchmal nicht auf den ersten Blick erkennbar ist und durch ihre Algorithmen eher im Hintergrund agiert. Die modernen digitalen Lehrmethoden und Sensortechnik eröffnen neue Wege für die Einbindung der KI in die chirurgische Aus- und Weiterbildung. Die ersten Schritte sind bereits implementiert – KI unterstützt die Erstellung von digitalem Bildmaterial zu Lehrzwecken oder erfasst mithilfe von Sensortechnik die chirurgische Leistung, um diese zu analysieren. Diverse Virtual- und Augmented-Reality-Simulatoren werden nicht nur als effektive Trainingswerkzeuge genutzt, sondern stellen vielmehr wertvolle Quellen für chirurgische Daten dar. All dies birgt enormes Potenzial für neue Erkenntnisse im Bereich der Lehrforschung, was zur Entwicklung von hocheffektiven und evidenzbasierten Lehrkonzepten beiträgt.

show abstract

The System for Telementoring with Augmented Reality (STAR): A head-mounted display to improve surgical coaching and confidence in remote areas

Cited by 53 publications

References 32 publications

Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

Augmented Reality in Medical Practice: From Spine Surgery to Remote Assistance

Smart glasses display device for fluoroscopically guided minimally invasive spinal instrumentation surgery: a preliminary study

Machine Learning and Learning Machines: KI in der Aus- und Weiterbildung

Contact Info

Product

Resources

About