On-patient see-through augmented reality based on visual SLAM

Mahmoud, Nader; Grasa, Óscar G.; Nicolau, Stéphane; Doignon, Christophe; Soler, Luc; Marescaux, Jacques; Montiel, José María

doi:10.1007/s11548-016-1444-x

Cited by 45 publications

(39 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These specifications helped increase the speed of algorithm processing, as well as yielded quality and stable video streams. The projected registration error of the proposed equation is around 2.5 ± 1.5 mm relative to the 5 mm observed by Mahmoud and colleagues and a processing time of 40 ± 20 ms/frame relative to the 66.2 ms/frame observed by Mahmoud et al Our proposed EIbAR system should, therefore, not only eliminate the need for interaction that limited the Mahmoud et al system, but also be more accurate and faster than current IbR systems.…”

Section: Proposed Systemmentioning

confidence: 61%

“…The details are mentioned in the Table . We used these samples to compare the performance of our proposed system to that of Mahmoud et al Comparison of the two systems was based on two factors: accuracy in terms of image registration error in millimetres (mm), and processing time in terms of milliseconds per frame (ms/frame). Accuracy in terms of registration error is reflective of the Root Median Square Distance (RMSD between the two images (pre‐operative and intra‐operative) using the real‐world ( x , y ) coordinates.…”

Section: Resultsmentioning

confidence: 99%

“…Evaluation of this system revealed that it has a registration error of 5 mm and processing time of on average 66.2 ms/frames, thereby indicating that Mahmoud and colleagues developed a system that can successfully and efficiently conduct image‐based registration between detailed 3D organ models built from CT data, which, consequently, enabled AR‐based surgical guidance. The strengths of this VSLaM system are not only that it supports a surgeon in viewing the locus of occluded sub‐surface structures, such as tumors, in the laparoscopic images, but that it does not: (a) disrupt surgical workflow; (b) require extensions to current laparoscopic hardware; or (c) need tracking sensors.…”

Section: State Of Art Systemmentioning

confidence: 99%

“…This figure depicts the workflow of our proposed EIbAR system. Green boxes specify the new proposed features and blue boxes specifies the existing features from Figure 2 Mahmoud et al…”

Section: Proposed Systemmentioning

confidence: 99%

“…This figure depicts the Mahmoud et al interactive VSLaM approach to AR visualization and localization of the liver. The red borders depict limitations of the system whereas the blue borders depict strengths of the system…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A novel enhanced intensity‐based automatic registration: Augmented reality for visualization and localization cancer tumors

Tan

Alsadoon

Prasad

et al. 2020

Robotics Computer Surgery

View full text Add to dashboard Cite

The purpose of this study is to replace the manual process (selecting the landmarks on mesh and anchor points on the video) by Intensity‐based Automatic Registration method to reach registration accuracy and low processing time. The proposed system consists of an Enhanced Intensity‐based Automatic Registration (EIbAR) using Modified Zero Normalized Cross Correlation (MZNCC) algorithm. The proposed system was implemented on videos of breast cancer tumors. Results showed that the proposed algorithm—as compared to a reference—improved registration accuracy by an average of 2 mm. In addition, the proposed algorithm—as compared to a reference—reduced the number of pixel matching, thereby reducing processing time on the video by an average of 22 ms/frame. The proposed system can, thus, provide an acceptable accuracy and processing time during scene augmentation of videos, which provides a seamless use of augmented‐reality for surgeons in visualizing cancer tumors.

show abstract

Section: Proposed Systemmentioning

confidence: 61%

Section: Resultsmentioning

confidence: 99%

Section: State Of Art Systemmentioning

confidence: 99%

“…This figure depicts the workflow of our proposed EIbAR system. Green boxes specify the new proposed features and blue boxes specifies the existing features from Figure 2 Mahmoud et al…”

Section: Proposed Systemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A novel enhanced intensity‐based automatic registration: Augmented reality for visualization and localization cancer tumors

Tan

Alsadoon

Prasad

et al. 2020

Robotics Computer Surgery

View full text Add to dashboard Cite

show abstract

Development of augmented‐reality applications in otolaryngology–head and neck surgery

et al. 2019

View full text Add to dashboard Cite

Objectives/Hypothesis: Augmented reality (AR) allows for the addition of transparent virtual images and video to one's view of a physical environment. Our objective was to develop a head-worn, AR system for accurate, intraoperative localization of pathology and normal anatomic landmarks during open head and neck surgery.Study Design: Face validity and case study. Methods: A protocol was developed for the creation of three-dimensional (3D) virtual models based on computed tomography scans. Using the HoloLens AR platform, a novel system of registration and tracking was developed. Accuracy was determined in relation to actual physical landmarks. A face validity study was then performed in which otolaryngologists were asked to evaluate the technology and perform a simulated surgical task using AR image guidance. A case study highlighting the potential usefulness of the technology is also presented.Results: An AR system was developed for intraoperative 3D visualization and localization. The average error in measurement of accuracy was 2.47 AE 0.46 millimeters (1.99, 3.30). The face validity study supports the potential of this system to improve safety and efficiency in open head and neck surgical procedures.Conclusions: An AR system for accurate localization of pathology and normal anatomic landmarks of the head and neck is feasible with current technology. A face validity study reveals the potential value of the system in intraoperative image guidance. This application of AR, among others in the field of otolaryngology-head and neck surgery, promises to improve surgical efficiency and patient safety in the operating room.

show abstract

Augmented reality for narrow area navigation in jaw surgery: Modified tracking by detection volume subtraction algorithm

Budhathoki

Alsadoon

Prasad

et al. 2020

Robotics Computer Surgery

View full text Add to dashboard Cite

Background and Aim: Jaw surgery based on augmented reality (AR) still has limitations in terms of navigating narrow areas. Surgeons need to avoid nerves, vessels, and teeth in their entirety, not just root canals. Inaccurate positioning of the surgical instrument may lead to positional or navigational errors and can result in cut blood vessels, nerve channels, or root canals. This research aims to decrease the positional error during surgery and improve navigational accuracy by reducing the positional error.Methodology: The proposed 2D/3D system tracks the surgical instrument, consisting of the shaft and the cutting element, each part being assigned a different feature description.In the case of the 3D position estimation, the input vector is composed of image descriptors of the instrument and the output value consists of 3D coordinates of the cutter.Results: Sample results from a jawbone-maxillary and mandibular jaw-demonstrate that the positional error is reduced. The system, thus, led to an improvement in alignment of the video accuracy by 0.25 to 0.35 mm from 0.40 to 0.55 mm and a decrease in processing time of 11 to 14 frames per second (fps) against 8 to 12 fps of existing solutions.Conclusion: The proposed system is focused on overlaying only on the area to be operated on. Thus, this AR-based study contributes to accuracy in navigation of the deeper anatomical corridors through increased accuracy in positioning of surgical instruments.

show abstract

On-patient see-through augmented reality based on visual SLAM

Abstract: The proposed system can be smoothly integrated into the surgical workflow because it: (1) operates in real time, (2) requires minimal additional hardware only a tablet-PC with camera, (3) is robust to occlusion, (4) requires minimal interaction from the medical staff.

Cited by 45 publications

References 24 publications

A novel enhanced intensity‐based automatic registration: Augmented reality for visualization and localization cancer tumors

A novel enhanced intensity‐based automatic registration: Augmented reality for visualization and localization cancer tumors

Development of augmented‐reality applications in otolaryngology–head and neck surgery

Augmented reality for narrow area navigation in jaw surgery: Modified tracking by detection volume subtraction algorithm

Contact Info

Product

Resources

About