A Review of Augmented Reality in Robotic-Assisted Surgery

Qian, Long; Wu, Jie; DiMaio, Simon P.; Navab, Nassir; Kazanzides, Peter

doi:10.1109/tmrb.2019.2957061

Cited by 83 publications

(40 citation statements)

References 123 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Since Boeing researcher Tom Caudell first proposed the term of AR in 1990 [ 6 ], the concept of AR has been gradually accepted. In 1997, Azuma et al defined the AR with three technical points [ 7 ]: (1) Combination of real and virtual views; (2) Real-time interactions; (3) Views registered in 3D.…”

Section: Introductionmentioning

confidence: 99%

A Projector-Based Augmented Reality Navigation System for Computer-Assisted Surgery

Gao

Zhao

Xie

et al. 2021

Sensors

View full text Add to dashboard Cite

In the medical field, guidance to follow the surgical plan is crucial. Image overlay projection is a solution to link the surgical plan with the patient. It realizes augmented reality (AR) by projecting computer-generated image on the surface of the target through a projector, which can visualize additional information to the scene. By overlaying anatomical information or surgical plans on the surgery area, projection helps to enhance the surgeon’s understanding of the anatomical structure, and intuitively visualizes the surgical target and key structures of the operation, and avoid the surgeon’s sight diversion between monitor and patient. However, it still remains a challenge to project the surgical navigation information on the target precisely and efficiently. In this study, we propose a projector-based surgical navigation system. Through the gray code-based calibration method, the projector can be calibrated with a camera and then be integrated with an optical spatial locator, so that the navigation information of the operation can be accurately projected onto the target area. We validated the projection accuracy of the system through back projection, with average projection error of 3.37 pixels in x direction and 1.51 pixels in y direction, and model projection with an average position error of 1.03 ± 0.43 mm, and carried out puncture experiments using the system with correct rate of 99%, and qualitatively analyzed the system’s performance through the questionnaire. The results demonstrate the efficacy of our proposed AR system.

show abstract

Section: Introductionmentioning

confidence: 99%

A Projector-Based Augmented Reality Navigation System for Computer-Assisted Surgery

Gao

Zhao

Xie

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…In addition, console display of da Vinci system can be adapted to support AR techniques during the surgery obviating the need for additional displays for supporting AR during surgery. The state of the art in AR-integrated the Robot Assisted Surgery (RAS) was reviewed by Qian et al (2019b) [63]. A brief overview of AR applications developed within 5 year period in the area of medical robotics along with the corresponding list of technology utilized for the creation of AR setups, tested robotic systems, and observed overall limitations are summarized in Table 1.…”

Section: Ar In Medical Roboticsmentioning

confidence: 99%

Augmented Reality for Robotics: A Review

2020

View full text Add to dashboard Cite

Augmented reality (AR) is used to enhance the perception of the real world by integrating virtual objects to an image sequence acquired from various camera technologies. Numerous AR applications in robotics have been developed in recent years. The aim of this paper is to provide an overview of AR research in robotics during the five year period from 2015 to 2019. We classified these works in terms of application areas into four categories: (1) Medical robotics: Robot-Assisted surgery (RAS), prosthetics, rehabilitation, and training systems; (2) Motion planning and control: trajectory generation, robot programming, simulation, and manipulation; (3) Human-robot interaction (HRI): teleoperation, collaborative interfaces, wearable robots, haptic interfaces, brain-computer interfaces (BCIs), and gaming; (4) Multi-agent systems: use of visual feedback to remotely control drones, robot swarms, and robots with shared workspace. Recent developments in AR technology are discussed followed by the challenges met in AR due to issues of camera localization, environment mapping, and registration. We explore AR applications in terms of how AR was integrated and which improvements it introduced to corresponding fields of robotics. In addition, we summarize the major limitations of the presented applications in each category. Finally, we conclude our review with future directions of AR research in robotics. The survey covers over 100 research works published over the last five years.

show abstract

“…Das in der Orthopädie genutzte MAKO-System (Stryker, Kalamazoo, MI, USA) plant präoperativ virtuelle Grenzen, die ein am Roboter befestigter und zugleich manuell geführter Fräser nicht überwinden kann. Intuitive Schnittstellen zum Chirurgen bilden bei robotischen Navigationssystemen oft haptische Informationen oder erweiterte Realität, indem chirurgische Bilder auf dem Bildschirm augmentiert oder Informationen in Datenbrillen dargestellt werden [ 17 ].…”

Section: Robotische Assistenzunclassified

Robotik im Operationssaal – (Ko‑)Operieren mit Kollege Roboter

Mathis-Ullrich

Scheikl

2020

Gastroenterologe

View full text Add to dashboard Cite

Hintergrund Medizinrobotik birgt das Potenzial, chirurgische oder endoluminale Eingriffe zu verbessern, indem diese Technologie hochpräzise Bewegungen und übermenschliche Perzeption ermöglicht. Ziel der Arbeit Darstellung historischer, existierender und zukünftiger robotischer Assistenten sowie Herausstellung deren Auswirkungen auf die robotische Chirurgie und Endoskopie. Methoden Insbesondere werden historische Medizinroboter sowie konventionelle Telemanipulatoren vorgestellt und diese mit minimal-invasiven Kontinuumsrobotern und neuartigen robotischen Konzepten aus Praxis und Forschung verglichen. Zusätzlich wird eine Perspektive für die zukünftige Generation von Chirurgie- und Endoskopierobotern geboten. Schlussfolgerung Roboterassistierte Medizin bietet einen großen Mehrwert für die Qualität der Intervention sowie Sicherheit für Chirurgen und Patientinnen. Zukünftig werden vermehrt Prozessschritte (teil‑)autonom in Kooperation mit Experten durchgeführt.

show abstract

A Review of Augmented Reality in Robotic-Assisted Surgery

Cited by 83 publications

References 123 publications

A Projector-Based Augmented Reality Navigation System for Computer-Assisted Surgery

A Projector-Based Augmented Reality Navigation System for Computer-Assisted Surgery

Augmented Reality for Robotics: A Review

Robotik im Operationssaal – (Ko‑)Operieren mit Kollege Roboter

Contact Info

Product

Resources

About