Characterizing Exploratory Visual Analysis: A Literature Review and Evaluation of Analytic Provenance in Tableau

Abstract

Supporting exploratory visual analysis (EVA) is a central goal of visualization research, and yet our understanding of the process is arguably vague and piecemeal. We contribute a consistent definition of EVA through review of the relevant literature, and an empirical evaluation of existing assumptions regarding how analysts perform EVA using Tableau, a popular visual analysis tool. We present the results of a study where 27 Tableau users answered various analysis questions across 3 datasets. We measure task performance, identify recurring patterns across participants' analyses, and assess variance from task specificity and dataset. We find striking differences between existing assumptions and the collected data. Participants successfully completed a variety of tasks, with over 80% accuracy across focused tasks with measurably correct answers. The observed cadence of analyses is surprisingly slow compared to popular assumptions from the database community. We find significant overlap in analyses across participants, showing that EVA behaviors can be predictable. Furthermore, we find few structural differences between behavior graphs for open‐ended and more focused exploration tasks.

Characterizing the Quality of Insight by Interactions: A Case Study

“…Two such metrics of exploration uniqueness and exploration pacing were proposed by Feng et al [22]. Battle and Heer [4] found that users tend to plan and execute a few steps of their interactions at a time as an exploration pace.…”

Database Benchmarking for Supporting Real-Time Interactive Querying of Large Data

“…Interactive data exploration is considered in some ways synonymous with information visualization [9,66], a core tenet of which is to facilitate "interactive, visual representations of abstract data to amplify cognition" [12, p. 7]. Thus the manipulation of visualizations through interactions is considered necessary to uncovering useful insights [4,50].…”

“…This section describes methodology used for designing our user study and collecting interaction traces. Our study design is based directly on prior studies of exploratory data analysis contexts [8,37,72], such as the study conducted by Battle & Heer to analyze exploration behavior with Tableau [9], and the studies by Liu & Heer [37] and Zgraggen et al [72] to assess the effects of latency in exploratory data analysis.…”

“…As a final contribution, we have made all of our study traces, benchmark code, and analysis code available online as a community resource to support future research. 1 As observed in prior work (e.g., [9,20,31,40,52]), experiment data is a highly-valuable source of insights not only for the database community but for other communities as well. Hence, we see this as an important step towards collaboration between the database and visualization communities.…”

“…Motivation. The data science process often begins with users (i.e., analysts, data scientists) exploring possibly massive amounts of data through interactions with a graphical user interface, oftentimes a data visualization tool [4,9,12,66]. However, each time a user interacts with a visualization interface, the underlying data must be processed (filtered, aggregated, etc.)…”