Augmented Reality of the Middle Ear Combining Otoendoscopy and Temporal Bone Computed Tomography

Marroquin, Roberto; Lalande, Alain; Hussain, Raabid; Guigou, Caroline; Grayeli, Alexis Bozorg

doi:10.1097/mao.0000000000001922

Cited by 24 publications

(39 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The aim of this study was to develop and assess a real-time AR system combining CT-scan data and microscopic video of the ear canal together with visualization of the surgical instrument behind closed tympanic membrane. This article extends our previous studies on AR based transtympanic procedures 15,22 by (1) validating the system's tracking schemes in near-realistic and challenging scenarios, and (2) imitating an actual procedure (drug administration), and (3) assessing the work in real-time instead of employing a recorded video. These developments brought the system several steps closer to its application in the operating room.…”

mentioning

confidence: 60%

“…A robust estimation of the operative microscope movements solely based on image features was developed in order to maintain correspondence with the CT-scan image 22 . A tracking scheme, comprising of RANSAC and nearest neighbour based Speeded-Up Robust Features (SURF) matching process was employed to determine transformation between consecutive frames [36][37][38] .…”

Section: Microscope Movement Trackingmentioning

confidence: 99%

“…However, they are expensive and bulky. Alternatively, image-based algorithms exploiting optical flow and image features are being developed and incorporated into the AR setup 19,[22][23][24] . When optimized, these methods appear to be more ergonomic to apply with a simpler system setup.…”

mentioning

confidence: 99%

“…In ear surgery, the small size of the target structures requires submillimetric precision 30 . As a proof of concept, it has been shown that AR combining otoendoscopy video and CT-scan may provide this level of precision in the middle ear if a careful registration is manually conducted by an expert 22 .…”

mentioning

confidence: 99%

“…In a previous work, the applicability and performance of different tracking processes (using both endoscope and surgical microscope) was assessed on both human cadaveric temporal bones and artificial temporal bones 15,22 . The aim of this study was to develop and assess a real-time AR system combining CT-scan data and microscopic video of the ear canal together with visualization of the surgical instrument behind closed tympanic membrane.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Hussain¹,

Lalande²,

Marroquin³

et al. 2020

Sci Rep

Self Cite

View full text Add to dashboard Cite

the aim of the study was to develop and assess the performance of a video-based augmented reality system, combining preoperative computed tomography (ct) and real-time microscopic video, as the first crucial step to keyhole middle ear procedures through a tympanic membrane puncture. Six different artificial human temporal bones were included in this prospective study. Six stainless steel fiducial markers were glued on the periphery of the eardrum, and a high-resolution CT-scan of the temporal bone was obtained. Virtual endoscopy of the middle ear based on this CT-scan was conducted on Osirix software. Virtual endoscopy image was registered to the microscope-based video of the intact tympanic membrane based on fiducial markers and a homography transformation was applied during microscope movements. These movements were tracked using Speeded-Up Robust Features (SURF) method. Simultaneously, a micro-surgical instrument was identified and tracked using a Kalman filter. The 3D position of the instrument was extracted by solving a three-point perspective framework. For evaluation, the instrument was introduced through the tympanic membrane and ink droplets were injected on three middle ear structures. An average initial registration accuracy of 0.21 ± 0.10 mm (n = 3) was achieved with a slow propagation error during tracking (0.04 ± 0.07 mm). The estimated surgical instrument tip position error was 0.33 ± 0.22 mm. The target structures' localization accuracy was 0.52 ± 0.15 mm. The submillimetric accuracy of our system without tracker is compatible with ear surgery.

show abstract

mentioning

confidence: 60%

Section: Microscope Movement Trackingmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Hussain¹,

Lalande²,

Marroquin³

et al. 2020

Sci Rep

Self Cite

View full text Add to dashboard Cite

show abstract

Development of augmented‐reality applications in otolaryngology–head and neck surgery

et al. 2019

View full text Add to dashboard Cite

Objectives/Hypothesis: Augmented reality (AR) allows for the addition of transparent virtual images and video to one's view of a physical environment. Our objective was to develop a head-worn, AR system for accurate, intraoperative localization of pathology and normal anatomic landmarks during open head and neck surgery.Study Design: Face validity and case study. Methods: A protocol was developed for the creation of three-dimensional (3D) virtual models based on computed tomography scans. Using the HoloLens AR platform, a novel system of registration and tracking was developed. Accuracy was determined in relation to actual physical landmarks. A face validity study was then performed in which otolaryngologists were asked to evaluate the technology and perform a simulated surgical task using AR image guidance. A case study highlighting the potential usefulness of the technology is also presented.Results: An AR system was developed for intraoperative 3D visualization and localization. The average error in measurement of accuracy was 2.47 AE 0.46 millimeters (1.99, 3.30). The face validity study supports the potential of this system to improve safety and efficiency in open head and neck surgical procedures.Conclusions: An AR system for accurate localization of pathology and normal anatomic landmarks of the head and neck is feasible with current technology. A face validity study reveals the potential value of the system in intraoperative image guidance. This application of AR, among others in the field of otolaryngology-head and neck surgery, promises to improve surgical efficiency and patient safety in the operating room.

show abstract

Augmented Reality in Otology/Neurotology: A Scoping Review with Implications for Practice and Education

Chen

Ding

et al. 2022

The Laryngoscope

View full text Add to dashboard Cite

Objective To determine how augmented reality (AR) has been applied to the field of otology/neurotology, examine trends and gaps in research, and provide an assessment of the future potential of this technology within surgical practice and education. Data Sources PubMed, EMBASE, and Cochrane Library were assessed from their inceptions through October 2022. A manual bibliography search was also conducted. Review Methods A scoping review was conducted and reported according to PRISMA‐ScR guidelines. Data from studies describing the application of AR to the field of otology/neurotology were evaluated, according to a priori inclusion/exclusion criteria. Exclusion criteria included non‐English language articles, abstracts, letters/commentaries, conference papers, and review articles. Results Eighteen articles covering a diverse range of AR platforms were included. Publication dates spanned from 2007 to 2022 and the rate of publication increased over this time. Six of 18 studies were case series in human patients although the remaining were proof of concepts in cadaveric/artificial/animal models. The most common application of AR was for surgical navigation (14 of 18 studies). Computed tomography was the most common source of input data. Few studies noted potential applications to surgical training. Conclusion Interest in the application of AR to otology/neurotology is growing based on the number of recent publications that use a broad range of hardware, software, and AR platforms. Large gaps in research such as the need for submillimeter registration error must be addressed prior to adoption in the operating room and for educational purposes. Level of Evidence N/A Laryngoscope, 133:1786–1795, 2023

show abstract

Augmented Reality of the Middle Ear Combining Otoendoscopy and Temporal Bone Computed Tomography

Abstract: A precise augmented reality combining video and 3D CT-scan data can be applied to otoendoscopy without the use of conventional neuronavigation tracking thanks to computer vision algorithms.

Cited by 24 publications

References 24 publications

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Video-based augmented reality combining CT-scan and instrument position data to microscope view in middle ear surgery

Development of augmented‐reality applications in otolaryngology–head and neck surgery

Augmented Reality in Otology/Neurotology: A Scoping Review with Implications for Practice and Education

Contact Info

Product

Resources

About