Visual Cue Based Corrective Feedback for Motor Skill Training in Mixed Reality: A Survey

Abstract

When learning a motor skill it is helpful to get corrective feedback from an instructor. This will support the learner to execute the movement correctly. With modern technology, it is possible to provide this feedback via mixed reality. In most cases, this involves visual cues to help the user understand the corrective feedback. We analyzed recent research approaches utilizing visual cues for feedback in mixed reality. The scope of this paper is visual feedback for motor skill learning, which involves physical therapy, exercise, rehabilitation etc. While some of the surveyed literature discusses therapeutic effects of the training, this paper focuses on visualization techniques. We categorized the literature from a visualization standpoint, including visual cues, technology and characteristics of the feedback. This provided insights into how visual feedback in mixed reality is applied in the literature and how different aspects of the feedback are related. The insights obtained can help to better adjust future feedback systems to the target group and their needs. This paper also provides a deeper understanding of the characteristics of the visual cues in general and promotes future, more detailed research on this topic.

Design Space of Visual Feedforward And Corrective Feedback in XR-Based Motion Guidance Systems

“…The start set of our snowball comprises three logical subsets. Start set A contains two comprehensive survey papers: one on corrective feedback cues by Diller et al [13], and one on rehabilitation exercises in XR by Butz et al [5]. Start set B contains prominent publications (over 100 citations on Google Scholar) that fit our scope: Just Follow Me [58], ShadowGuides [18], LightGuide [51], YouMove [1] and Physio@Home [53].…”

“…We use this corpus to devise design space for motion guidance in terms of its two components: motion feedforward (Section 4) and corrective feedback (Section 5). We first extracted initial design dimensions from existing literature [3,11,13]. We then augmented these dimensions by examining the many different technical setups present in our corpus, drawing on our own experience in the fields of XR and motion guidance.…”

“…Most relevant to our work is the survey by Diller et al [13] of visual corrective feedback cues in XR regarding motion training. They categorized visual corrective feedback in existing motion training designs based on nine attributes: MR technologies, point of view, abstraction type, temporal order, stages of learning, publication venue, body parts, use case, and visual cues.…”

“…Hence, this phenomenon has been regarded as a form of corrective feedback within a number of related literature. In the survey by Diller et al [13], the visual cues classified as "end position", "transparent target avatar", "opaque target avatar", "video overlay", "body outline", "movement abstraction" and "trajectory" all fall into this group, with the first five cues manifesting themselves through postural differences, while the last two reflecting trajectorial differences. Likewise, systems that use continuous [22] or discrete [18] guidance naturally indicate errors due to the need to align the trainee's body to the required checkpoint.…”

“…Hence, we structure our investigation following the same two components. Recently, Diller et al [13] conducted a related survey to summarize the work of using XR to provide corrective feedback for motor training. However, their survey focuses only on corrective feedback, and does not consider the relationship between it and the presented motion feedforward.…”