A Multimodal Natural Interaction-Based Mixed Reality System for Limb Rehabilitation

Shou, Kailiang; Meng, Qianyu; Yuan, Qingshu; Yang, Qihao; Pan, Zhigeng

doi:10.1109/icvr57957.2023.10169638

Cited by 1 publication

(2 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This enhances the authenticity, naturalness, and efficiency of the user interaction process. Shou et al [53] proposed a multimodal natural interaction technology based on gestures, tangible objects, and natural walking, designed for patients with limb impairments. User research indicated that compared to controller interaction, natural multimodal interaction is more engaging, provides a stronger sense of realism, and enhances participants' enthusiasm and motivation.…”

Section: Multimodal Interactionmentioning

confidence: 99%

“…Moreover, gaze and voice work more effectively together, which does not distract the user spatially, as voice and gaze may be directed at the same information recipient, suitable for quickly indicating where or on which object an action should be applied [75]. These three interaction modalities work together since students are more likely to develop meaningful learning in environments with multimodal interaction [50], contributing to heightened user enthusiasm [53].…”

Section: Multimodal Interaction Modulementioning

confidence: 99%

See 1 more Smart Citation

Examining the impact of head‐mounted augmented reality on learning engineering drawings: A case study for three‐view drawing

Yuan,

Shou,

et al. 2024

Comp Applic In Engineering

Self Cite

View full text Add to dashboard Cite

As the cornerstone of spatial geometry and engineering drawings, three‐view drawing lays the necessary foundation for students to learn geography, graphics, industrial design, geographic information science, and so forth. However, teaching three‐view drawing can be challenging due to its abstraction. Currently, desktop and handheld augmented reality (AR) systems are widely applied to three‐view teaching. However, existing systems suffer from visuo‐tactile inconsistency and contain unfriendly interactions, resulting in a poor learning experience. In this paper, head‐mounted AR is introduced for the first time in the teaching of three‐view drawing. A system was designed for learning three‐view drawing to facilitate students' free exploration and learning in lessons. Additionally, a comparative experiment was designed and conducted to investigate whether multimodal interaction and head‐mounted AR help students to better understand the relationship between concrete three‐dimensional (3D) objects and abstract two‐dimensional planes and the impact on subjective perceptions. The results demonstrated that the proposed system significantly improved learning performance. Furthermore, the proposed system heightened students' interest in learning, made reproducing 3D structures more intuitive, and students were more willing to use such systems in their future studies.

show abstract

Section: Multimodal Interactionmentioning

confidence: 99%