Projector‐based augmented reality system for interventional visualization inside MRI scanners

Mewes, André; Heinrich, Florian; Kägebein, Urte; Hensen, Bennet; Wacker, Frank; Hansen, Christian

doi:10.1002/rcs.1950

Cited by 22 publications

(21 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…53 Nearly one-third of all studies (n=8) either had no evaluation tool or did not report any validity of the instrument used. [28][29][30][31][32][33][34][35] The qualitative study involved semi-structured face-toface interviews that explored the needs and challenges of applying AR for healthcare education. The study demonstrated a detailed clarity and rigor according to the individual AQRAME score of all three authors corresponding to 12 (JG), 11.5 (CBS), and 12 (PD).…”

Section: Methodological Qualitymentioning

confidence: 99%

“…30,31 Four studies recognized a virtual model registered with a phantom characterized as MR. 27,[50][51][52] Eleven studies reported using external cameras and tracking devices. 27,28,31,32,34,36,44,[50][51][52] Two studies used applications based on projectors, one recognizing markers on a phantom, and one projecting images directly onto a phantom without using a tracking device. 29,51…”

Section: Technical Specificationsmentioning

confidence: 99%

See 1 more Smart Citation

Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review

2020

View full text Add to dashboard Cite

Objective: This study aimed to review and synthesize the current research and state of augmented reality (AR), mixed reality (MR) and the applications developed for healthcare education beyond surgery. Methods: An integrative review was conducted on all relevant material, drawing on different data sources, including the databases of PubMed, PsycINFO, and ERIC from January 2013 till September 2018. Inductive content analysis and qualitative synthesis were performed. Additionally, the quality of the studies was assessed with different structured tools. Results: Twenty-six studies were included. Studies based on both AR and MR involved established applications in 27% of all cases (n=6), the rest being prototypes. The most frequently studied subjects were related to anatomy and anesthesia (n=13). All studies showed several healthcare educational benefits of AR and MR, significantly outperforming traditional learning approaches in 11 studies examining various outcomes. Studies had a low-to-medium quality overall with a MERSQI mean of 12.26 (SD=2.63), while the single qualitative study had high quality. Conclusion: This review suggests the progress of learning approaches based on AR and MR for various medical subjects while moving the research base away from feasibility studies on prototypes. Yet, lacking validity of study conclusions, heterogeneity of research designs and widely varied reporting challenges transferability of the findings in the studies included in the review. Future studies should examine suitable research designs and instructional objectives achievable by AR and MR-based applications to strengthen the evidence base, making it relevant for medical educators and institutions to apply the technologies.

show abstract

Section: Methodological Qualitymentioning

confidence: 99%

Section: Technical Specificationsmentioning

confidence: 99%

Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review

2020

View full text Add to dashboard Cite

show abstract

“…Projector‐camera systems (Figure ) have been extensively used as a hardware configuration for the table top interface for various applications in media art, education, and entertainment as they effectively visualize information in space and allow users to manipulate visual entities intuitively . Applied in augmented reality (AR), the system realizes the projection‐based AR and can visualize relevant information on user's hand and objects …”

Section: Introductionmentioning

confidence: 99%

“…A key component for the tangible object‐based interaction is the object classification to identify the physical objects. For this, a number of features are used, such as shape, color of an object, or specific markers attached to the object . While appropriate features need to be selected according to the characteristics of the system, color is a useful feature especially when multiple objects have the same shape but different colors.…”

Section: Introductionmentioning

confidence: 99%

“…[1][2][3][4][5][6][7][8] Applied in augmented reality (AR), the system realizes the projection-based AR and can visualize relevant information on user's hand and objects. [9][10][11] In the projector-camera system, user interaction is implemented in many ways. A method using multitouch displays allows users to directly manipulate virtual entities by fingers without using additional devices.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Color refinement using deep neural networks for enhancing color recognition in a projector‐camera system

2019

View full text Add to dashboard Cite

In projector‐camera systems, object recognition is essential to enable users to interact with physical objects. Among several input features used by the object classifier, color information is widely used as it is easily obtainable. However, the color of an object seen by the camera changes due to the projected light from the projector, which degrades the recognition performance. To solve this problem, we propose a method to restore the original color of an object from the observed color through camera. The color refinement method has been developed based on the deep neural network. The inputs to the neural network are the color of the projector light as well as the observed color of the object in multiple color spaces, including RGB, HSV, HIS, and HSL. The neural network is trained in a supervised manner. Through a number of experiments, we show that our refinement method reduces the difference from the original color and improves the object recognition rate implemented with a number of classification methods.

show abstract

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Velazco-Garcia

Navkar

Balakrishnan

et al. 2021

Robotics Computer Surgery

View full text Add to dashboard Cite

Background: User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment. Method:End-user studies were conducted by simulating an MRgPBx system with end-and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system. Results:The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system. Conclusions:The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning.

show abstract

Projector‐based augmented reality system for interventional visualization inside MRI scanners

Cited by 22 publications

References 34 publications

Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review

Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review

Color refinement using deep neural networks for enhancing color recognition in a projector‐camera system

Evaluation of how users interface with holographic augmented reality surgical scenes: Interactive planning MR‐Guided prostate biopsies

Contact Info

Product

Resources

About