Pedagogy and usability in interactive algorithm visualizations: Designing and evaluating CIspace

Abstract

Interactive algorithm visualizations (AVs) are powerful tools for teaching and learning concepts that are difficult to describe with static media alone. However, while countless AVs exist, their widespread adoption by the academic community has not occurred due to usability problems and mixed results of pedagogical effectiveness reported in the AV and education literature. This paper presents our experiences designing and evaluating CIspace, a set of interactive AVs for demonstrating fundamental Artificial Intelligence algorithms. In particular, we first review related work on AVs and theories of learning. Then, from this literature, we extract and compile a taxonomy of goals for designing interactive AVs that address key pedagogical and usability limitations of existing AVs. We advocate that differentiating between goals and design features that implement these goals will help designers of AVs make more informed choices, especially considering the abundance of often conflicting and inconsistent design recommendations in the AV literature. We also describe and present the results of a range of evaluations that we have conducted on CIspace that include semi-formal usability studies, usability surveys from actual students using CIspace as a course resource, and formal user studies designed to assess the pedagogical effectiveness of CIspace in terms of both knowledge gain and user preference. Our main results show that (i) studying with our interactive AVs is at least as effective at increasing student knowledge as studying with carefully designed paper-based materials; (ii) students like using our interactive AVs more than studying with the paper-based materials; (iii) students use both our interactive AVs and paper-based materials in practice although they are divided when forced to choose between them; (iv) students find our interactive AVs generally easy to use and useful. From these results, we conclude that while interactive AVs may not be universally preferred by students, it is beneficial to offer a variety of learning media to students to accommodate individual learning preferences. We hope that our experiences will be informative for other developers of interactive AVs, and encourage educators to exploit these potentially powerful resources in classrooms and other learning environments.

Providing Adaptive Support in an Interactive Simulation for Learning

“…The test-bed for our research is the AIspace CSP applet [1], an interactive simulation that illustrates the workings of the AC3 algorithm for constraint satisfaction problems. This applet is part of AIspace, a publicly available suite of interactive simulations for a variety of AI algorithms that is routinely used in AI courses both in our university and elsewhere.…”

“…This applet is part of AIspace, a publicly available suite of interactive simulations for a variety of AI algorithms that is routinely used in AI courses both in our university and elsewhere. There is extensive positive feedback from students when these applets are used for in-class demonstrations, and formal studies on the CSP applet have shown that it both fosters learning and motivates students [1]. These studies, however, also showed that not all students leverage at best the functionalities available to explore the AC3 algorithm mechanics.…”

TreatmentExplorer: an Interactive Decision Aid for Medical Risk Communication and Treatment Exploration

“…to reveal trends over time or transformations of graphical representations during exploration (Chevalier et al, 2010;Heer and Robertson, 2007), but new examples and further user studies are needed to sharpen our understanding of which situations benefit from animation and which ones do not (Tversky, 2011;Amershi et al, 2008). Finally the design of TreatmentExplorer was inspired by the Lifeflow visualization, which summarizes temporal patterns of electronic health records (Wongsuphasawat et al, 2011).…”

Discovering and Recognizing Student Interaction Patterns in Exploratory Learning Environments

“…The ELE we use as a testbed for our approach is the Constraint Satisfaction Problem (CSP) Applet, one of a collection of interactive tools for learning common Artificial Intelligence algorithms, called AIspace [4]. Algorithm dynamics are demonstrated via interactive visualizations on graphs by the use of color and highlighting, and graphical state changes are reinforced through textual messages (see Figure 1 for an example).…”

“…The approach based on association rules and a class association rule classifier that we present in this paper has a finer classification granularity, and thus it is better suited at guiding adaptive interventions to improve interaction effectiveness. We test the approach on AISpace [4], an ELE that uses interactive visualizations to help students understand Artificial Intelligence (AI) algorithms.…”