Dream-Experiment: A MR User Interface with Natural Multi-channel Interaction for Virtual Experiments

Luo, Tianren; Zhang, Mingmin; Pan, Zhigeng; Li, Zheng; Cai, Ning; Miao, Jinda; Chen, Youbin; Xu, Mingxi

doi:10.1109/tvcg.2020.3023602

Cited by 20 publications

(10 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Physicians use AR/VR technology to gain a deeper understanding of surgical risks and surgical plans, reduce differences in patient-patient understanding, and reduce physician-patient inconsistencies caused by unequal medical knowledge and information [ 22 ]. As shown in Figure 1 , VR technology can simulate the human body.…”

Section: Vr/ar Technology and Value In Teachingmentioning

confidence: 99%

VR/AR Technology in Human Anatomy Teaching and Operation Training

Zhou

Hou

Liu

et al. 2021

Journal of Healthcare Engineering

View full text Add to dashboard Cite

AR/VR technology can fuse the clinical imaging data and information to build an anatomical environment combining virtual and real, which is helpful to improve the interest of teaching and the learning initiative of medical students, and then improve the effect of clinical teaching. This paper studies the application and learning effect of the VR/AR system in human anatomy surgery teaching. This paper first shows the learning environment and platform of the VR/AR system, then explains the interface and operation of the system, and evaluates the teaching situation. This paper takes the VR/AR operation simulation system of an Irish company as an example and evaluates the learning effect of 41 students in our hospital. Research shows that the introduction of the feature reweighting module in the VR/AR surgery simulation system improves the accuracy of bone structure segmentation (IOU value increases from 79.62% to 83.56%). For real human ultrasound image data, the IOU value increases from 80.21% to 82.23% after the feature reweighting module is introduced. Therefore, the dense convolution module and feature reweighting module improve the learning ability of the network for bone structure features in ultrasound images from two aspects of feature connection and importance understanding and effectively improve the performance of bone structure segmentation.

show abstract

Section: Vr/ar Technology and Value In Teachingmentioning

confidence: 99%

VR/AR Technology in Human Anatomy Teaching and Operation Training

Zhou

Hou

Liu

et al. 2021

Journal of Healthcare Engineering

View full text Add to dashboard Cite

show abstract

“…The most common are voice, touch, and gestures. Voice input [11,25,26,30,38,39,47,49,50,51,55,56,57,58,59,60] can be command-based or natural language processing (NLP) [56]. Touch interaction [39,46,48,51,53,56,57,59,60,61] can include both near and far touch.…”

Section: D3 Input Modalitiesmentioning

confidence: 99%

“…Voice input [11,25,26,30,38,39,47,49,50,51,55,56,57,58,59,60] can be command-based or natural language processing (NLP) [56]. Touch interaction [39,46,48,51,53,56,57,59,60,61] can include both near and far touch. Near touch in this context refers to directly touching the interaction trigger, whereas far touch refers to a mapping of the touch interaction, i.e., ray-casting on a distant object [56].…”

Section: D3 Input Modalitiesmentioning

confidence: 99%

“…The authors describe an AR solution in which 3D content is attached and mapped onto the body, in this case around the user's wrist. Gestures [11,25,26,27,29,30,38,42,47,48,50,51,52,53,55,56,57,58,59,60,62,63] are the most commonly used AR input modality and include gestures using fingers, hands, arms or the entire body. They are widely used regardless of hardware, but tracking mechanisms are required to detect the gestures.…”

Section: D3 Input Modalitiesmentioning

confidence: 99%

See 1 more Smart Citation

Conceptualizing Interactions of Augmented Reality Solutions

Bräker¹,

Hertel²,

Semmann³

2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

The rapid evolution of augmented reality has resulted in an ever-increasing number of applications in a wide range of industries and services. Despite this progress, there is still a lack of conceptual understanding of AR interactions and the entire solution space. To bridge this gap, we conceptualize AR solution interactions and provide a comprehensive taxonomy. To represent the state-of-the-art, we build upon an extensive literature review. The resulting taxonomy consists of seven dimensions that encompass 29 characteristics. We contribute to the understanding of AR interactions and, as a result, the applicability of AR solutions in businesses by developing the taxonomy. Likewise, the taxonomy can guide the design of AR solutions as it convincingly describes the solution space.

show abstract

“…Real chemical experiments may be dangerous, especially with flames, corrosion, toxic gases, etc. Guex et al (2017), which will be difficult to meet the security needs in MHN theory, causing users to have psychological pressure Luo et al (2020). In addition, in many schools, due to the lack of teacher resources Ángel (2015), Martin-Villalba et al (2012), students often encounter problems hard to be solved, which is difficult to meet the needs of self-actualization.…”

Section: Introductionmentioning

confidence: 99%

MagicChem: a MR system based on needs theory for chemical experiments

et al. 2021

Self Cite

View full text Add to dashboard Cite

Real chemical experiments may be dangerous or pollute the environment; meanwhile, the preparation of drugs and reagents is time-consuming. Due to the above-mentioned reasons, few experiments can be actually operated by students, which is not conducive to the chemistry learning and the phenomena principle understanding. Recently, due to the impact of Covid-19, many schools adopt online teaching, which is even more detrimental to students' learning of chemistry. Fortunately, MR(mixed reality) technology provides us with the possibility of solving the safety issues and breaking the space-time constraints, while the theory of human needs (Maslow's hierarchical needs) provides us with a way to design a comfortable and stimulant MR system with realistic visual presentation and interaction. The paper combines with the theory of human needs to propose a new needs model for virtual experiment. Based on this needs model, we design and develop a comprehensive MR system called MagicChem, which offers a robust 6-DoF interactive and illumination consistent experimental space with virtual-real occlusion, supporting realistic visual interaction, tangible interaction, gesture interaction with touching, voice interaction, temperature interaction, olfactory interaction and virtual human interaction. User study shows that MagicChem satisfies the needs model better than other MR experimental environments that partially meet the needs model. In addition, we explore the application of the needs model in VR environment.

show abstract

Dream-Experiment: A MR User Interface with Natural Multi-channel Interaction for Virtual Experiments

Cited by 20 publications

References 44 publications

VR/AR Technology in Human Anatomy Teaching and Operation Training

VR/AR Technology in Human Anatomy Teaching and Operation Training

Conceptualizing Interactions of Augmented Reality Solutions

MagicChem: a MR system based on needs theory for chemical experiments

Contact Info

Product

Resources

About