User-adaptive explanatory program visualization: evaluation and insights from eye movements

Abstract

User-adaptive visualization and explanatory visualization have been suggested to increase educational effectiveness of program visualization. This paper presents an attempt to assess the value of these two approaches. The results of a controlled experiment indicate that explanatory visualization allows students to substantially increase the understanding of a new programming topic. Furthermore, an educational application that features explanatory visualization and employs a user model to track users' progress allows students to interact with a larger amount of material than an application which does not follow users' activity. However, no support for the difference in short-term knowledge gain between the two applications is found. Nevertheless, students admit that they prefer the version that estimates and visualizes their progress and adapts the learning content to their level of understanding. They also use the application's estimation to pace their work. The differences in eye movement patterns between the applications employing adaptive and non-adaptive explanatory visualizations are investigated as well. Gaze-based measures show that adaptive visualization captivates attention more than its non-personalized counterpart and is more interesting to students. Natural language explanations also accumulate a big portion of students' attention. Furthermore, the results indicate that working memory span can mediate the perception of adaptation. It is possible that user-adaptation in an educational context provides a different service to people with different mental processing capabilities.

Understanding Attention to Adaptive Hints in Educational Games: An Eye-Tracking Study

“…Others have used eye-tracking data to study attention to relevant components of an ITS, e.g. to Open Learner Models (Bull et al 2007;Mathews et al 2012), to adaptive animations (Loboda and Brusilovsky 2010) and, most related to our work, to ITS feedback messages (Gluck et al 2000). The distinguishing feature of our work is that we perform a more detailed analysis of which factors affect user attention to adaptive interventions, as well as whether differences in attention affect performance with the system.…”

The impact of user- and system-initiated personalization on the user experience at large sports events

“…Although personalization (in its broadest sense) undoubtedly provides benefits, the evidence is not wholly unequivocal. For example, Loboda and Brusilovsky (2010) investigated educational applications employing personalized and non-personalized explanatory visualizations.…”

Adaptive Semantics Visualization

“…Further they identified some more difficult concepts that are highlighted with orange flags. Explanation is one main functionality of ( I ) ( I I ) the system that has more the characteristics of an intelligent tutoring system [LB10]. Based on the degree of knowledge the student gets either a short or long explanation of the step to perform [BL06].…”

“…Brusilovsky and Su introduced with their WADEIn an adaptive visualization environment for expression execution as learning environment for the programming language C [BS02]. This system was enhanced with explanatory characteristics by Brusilovsky and Loboda[BL06] (WADEIn II ) and further enhanced (cWADEIn) and evaluated[LB08,LB10]. We introduce their approach based on the latest version that was described sufficiently and the core of their work, namely WADEIn II and cWADEIn [BL06, LB08, LB10] and use the term WADEIn II.WADEIn II is an adaptive explanatory visualization that enhances the model-based approach for generating explanation proposed byKumar [Kum03] with adaptive visualization approaches[BL06,LB08].…”