Feedback Design in Multimodal Dialogue Systems

Abstract

Abstract:This paper discusses the design and development of the instructional aspects of a multimodal dialogue system to train youth parliament members' presentation and debating skills. Real-time, in-action feedback informs learners on the fly how they perform key skills and enables them to adapt instantly. About-action feedback informs learners after finishing a task how they perform key skills and enables them to monitor their progress and adapt accordingly in subsequent tasks. In-and about-action feedback together support the enhancement of the learners' metacognitive skills, such as self-monitoring, self-regulation and selfreflection thus reflect in-and about action. We discusses the theoretical considerations of the feedback, the type of data available and different ways to analyse and combine them, the timing of feedback and, finally, provide an instructional design blueprint giving a global outline of a set of tasks with stepwise increasing complexity and the feedback proposed. We conclude with the results of the first experiment with the system focussing on non-verbal communication skills.

Do You Know What Your Nonverbal Behavior Communicates? – Studying a Self-reflection Module for the Presentation Trainer

“…The module added fits well within theories of reflection [22]. With the added module the PT enables now both reflection-in-action (reflection on behavior as it happens, so as to optimize the immediately following action) and reflection-about-action (reflection after the event, to review, analyze, and evaluate the situation, so as to gain insight for improved practice in future) [23].This evaluation allowed us to draw the following conclusions:…”

Observing, Coaching and Reflecting: Metalogue - A Multi-modal Tutoring System with Metacognitive Abilities

“…More specifically, these important nonverbal communication aspects include gaze and facial expressions [3], head movements and orientation [17,22], hand and arm gestures [25], and posture shifts or movements [42]. In the Metalogue project context the capturing and provisioning of feedback based upon these aspects has been outlined in [52] and further explored in [43].…”

“…A key feature of Metalogue is its ability to provide two different forms of feedback designed to enhance the learner's experience [52] based on Donald Schön's [45] distinction between reflection in-action and aboutaction. Schön defines reflective practice as the practice by which professionals become aware of their implicit knowledge base and learn from their experience.…”

“…For this reason, in-action feedback is currently envisioned in the form of a small number of simple signs and symbols displayed on-screen [43]. The goal of the given feedback is to help the learner become aware of their strengths and weaknesses, to enable a learner "to understand which aspects are of relevance and, ultimately, be able to recognise these aspects in their performance or the performance of others" [52]. While the in-action feedback is appropriately constrained, the about-action feedback provides the full range of possible feedback via a reflection dashboard that enables both tutor and learner to review and analyse the performance moment-by moment.…”