Virtual annotations of the surgical field through an augmented reality transparent display

Andersen, Daniel; Popescu, Voicu; Cabrera, María Eugenia; Shanghavi, Aditya; Gómez, Gerardo; Marley, Sherri; Mullis, Brian; Wachs, Juan P.

doi:10.1007/s00371-015-1135-6

Cited by 34 publications

(11 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, the AR users were significantly slower in six experiments due to limitations related to the systems. This may have been due to the delay in the network, video and audio signals (Ponce, Menendez et al, 2014;Wang et al, 2017); the narrow field of view of the HMD; the distorted correspondence between the displayed view and the real view and the deteriorated sense of distance (Oyama et al, 2010;Wang et al, 2017); or the deficiencies in hand-eye coordination due to the lack of a fully transparent effect on the display (Andersen et al, 2016). A longer time could also have been experienced as participants tried to match their operative field to the absolute 3D position in which the annotation was supposed to be located (Rojas-Munoz et al, 2018).…”

Section: Task Performancementioning

confidence: 99%

Tele-mentoring using augmented reality technology in healthcare: A systematic review

Bui

Barnett

Hoang

et al. 2021

AJET

View full text Add to dashboard Cite

This systematic review aimed to identify how tele-mentoring systems that incorporate augmented reality (AR) technology are being used in healthcare environments. A total of 12 electronic bibliographic databases were searched using the terms “augmented reality”, “tele-mentoring” and “health”. The PRISMA checklist was used as a guide for reporting. The mixed method appraisal tool was used to assess the quality of the included experiments. The data were then analysed using a concept-centric approach and categorised primarily with regards to system performance and task performance measures. A total of 11 randomised controlled trials and 14 non-randomised designs were included for review. Both mentees and mentors assessed the system and task performance according to 25 categories. The feedback of mentees using AR tele-mentoring systems was generally positive. The majority of experiments revealed that the AR system was an effective tele-mentoring device overall and resulted in the effective performance of a procedure. Benefits included improvements in trainees’ confidence, task completion time and reductions in task errors and shifts in focus. However, the systems had limitations, including heaviness of the equipment, inconvenience, discomfort and distraction of wearing devices, limited battery life, the latency of video and audio signals and limited field of view. Implications for practice or policy: Health practitioners can apply AR technology to receive and follow real-time annotated instructions verbally and visually from remote experts. Technical developers may consider improving AR devices in terms of lighter weight, larger field of view, more ergonomic design, more stable network connection and longer battery life. Further AR-related experiments may need to explore AR tele-mentoring systems’ utility across healthcare environments with larger samples, real patient populations in remote settings, cost-benefit analysis and impacts on short- and long-term patient outcomes.

show abstract

Section: Task Performancementioning

confidence: 99%

Tele-mentoring using augmented reality technology in healthcare: A systematic review

Bui

Barnett

Hoang

et al. 2021

AJET

View full text Add to dashboard Cite

show abstract

“…In this task, the surgeon and the nurse collaborate by delivering and retrieving surgical instruments to complete the surgery successfully. This simulation setup and the surgical task procedure have been used in our previous work (gesture robot nurse [36] and telementoring in the OR [37]).…”

Section: A Surgical Task Setupmentioning

confidence: 99%

Early prediction for physical human robot collaboration in the operating room

Zhou

Wachs

2017

Auton Robot

View full text Add to dashboard Cite

To enable a natural and fluent human robot collaboration flow, it is critical for a robot to comprehend their human peers' on-going actions, predict their behaviors in the near future, and plan its actions correspondingly. Specifically, the capability of making early predictions is important, so that the robot can foresee the precise timing of a turn-taking event and start motion planning and execution early enough to smooth the turn-taking transition. Such proactive behavior would reduce human's waiting time, increase efficiency and enhance naturalness in collaborative task. To that end, this paper presents the design and implementation of an early turn-taking prediction algorithm, catered for physical human robot collaboration scenarios. Specifically, a Robotic Scrub Nurse (RSN) system which can comprehend surgeon's multimodal communication cues and perform turn-taking prediction is presented. The developed algorithm was tested on a collected data set of simulated surgical procedures in a surgeon-nurse tandem. The proposed turn-taking prediction algorithm is found to be significantly superior to its algorithmic counterparts, and is more accurate than human baseline when little partial input is given (less than 30% of full action). After observing more information, the algorithm can achieve comparable performances as humans with a F1 score of 0.90.

show abstract

“…This approach can alleviate the problems arising from the loss of depth perception and obstructed vision encountered in MIS, by showing anatomical information of targets that would otherwise be hidden to the eye 51 52 86. Current setups have the possibility of offering ‘on-demand’ augmented reality overlaid directly on the video feed of the operative screen, using a transparent screen or a separate device such as a tablet PC 87 88. Current research is focusing on relieving the surgeon from the pressure of deciding when visual input is needed, employing machine learning algorithms and analysis of surgical workflows to automate and improve the augmented reality systems 89…”

Section: Imagingmentioning

confidence: 99%

The role of technology in minimally invasive surgery: state of the art, recent developments and future directions

Tonutti

Elson

Yang

et al. 2016

Postgraduate Medical Journal

View full text Add to dashboard Cite

The diffusion of minimally invasive surgery has thrived in recent years, providing substantial benefits over traditional techniques for a number of surgical interventions. This rapid growth has been possible due to significant advancements in medical technology, which partly solved some of the technical and clinical challenges associated with minimally invasive techniques. The issues that still limit its widespread adoption for some applications include the limited field of view; reduced manoeuvrability of the tools; lack of haptic feedback; loss of depth perception; extended learning curve; prolonged operative times and higher financial costs. The present review discusses some of the main recent technological advancements that fuelled the uptake of minimally invasive surgery, focussing especially on the areas of imaging, instrumentation, cameras and robotics. The current limitations of state-of-the-art technology are identified and addressed, proposing future research directions necessary to overcome them

show abstract

Virtual annotations of the surgical field through an augmented reality transparent display

Cited by 34 publications

References 23 publications

Tele-mentoring using augmented reality technology in healthcare: A systematic review

Tele-mentoring using augmented reality technology in healthcare: A systematic review

Early prediction for physical human robot collaboration in the operating room

The role of technology in minimally invasive surgery: state of the art, recent developments and future directions

Contact Info

Product

Resources

About