Many roads lead to Rome: Mapping users’ problem-solving strategies

Abstract

There is more than one path to a solution, especially when it comes to ill-defined problems like complex, realworld tasks. Until now, the evaluation of information visualizations has often been restricted simply to a measuring of outcomes (time and error) or insights into the data set. A more detailed look into the processes that facilitate or hinder task completion is provided by analysing user problem-solving strategies. The study presented in this paper illustrates how such processes can be assessed and how the resulting knowledge can be used in participatory design to improve a visual analytics tool. For users to be equipped with a problemsolving scaffold, the tools used should allow them to choose their own path to the solution -their own route to Rome. We also discuss how the evaluation of problem-solving strategies can shed more light on the 'exploratory minds' of users.

Evaluating user behavior and strategy during visual exploration

“…This way of looking at the analytic activity reflects a somewhat higher-level characterization compared to the low-level operators found by Amar et al [1]. However, this characterization also constitute a finer-grained analysis compared to existing qualitative evaluation methods (e.g., [8]). …”

“…To overcome this limitation, the evaluator could consider additional sources of data, such as verbal statements collected during interviews and think-aloud experiments, as well as by observing users' viewing behavior. Qualitative analysis here could provide a richer understanding of user strategy and ultimately lead to insights on how to improve the visualization tool [8]. In ill-defined exploratory tasks, however, there can be enormous variation in the high-level strategy of users, which can be sensitive to the dataset as well as differences in analytical abilities and problem-solving styles of individuals.…”

Towards analyzing eye tracking data for evaluating interactive visualization systems

“…The data was then represented in a visualization. An example used in this paper is from the study by Mayr et al [36] where two visualization prototypes for analyzing temporal granularity of data are compared. The authors wanted to analyze problemsolving strategies.…”

Mapping the Users’ Problem Solving Strategies in the Participatory Design of Visual Analytics Methods

“…Though insights are an outcome of cognitive processes during exploratory data analysis, they are not directly linked to the task at hand. To understand the users' cognitive processes while they are completing a task (or failing to do so) we proposed to analyze the problem solving processes [10]. Problems are the users' subjective representations of an objectively given task [11].…”