A Heuristic Approach to Value-Driven Evaluation of Visualizations

Abstract

To interpret data visualizations, people must determine how visual features map onto concepts. For example, to interpret colormaps, people must determine how dimensions of color (e.g., lightness, hue) map onto quantities of a given measure (e.g., brain activity, correlation magnitude). This process is easier when the encoded mappings in the visualization match people's predictions of how visual features will map onto concepts, their inferred mappings. To harness this principle in visualization design, it is necessary to understand what factors determine people's inferred mappings. In this study, we investigated how inferred color-quantity mappings for colormap data visualizations were influenced by the background color. Prior literature presents seemingly conflicting accounts of how the background color affects inferred color-quantity mappings. The present results help resolve those conflicts, demonstrating that sometimes the background has an effect and sometimes it does not, depending on whether the colormap appears to vary in opacity. When there is no apparent variation in opacity, participants infer that darker colors map to larger quantities (dark-is-more bias). As apparent variation in opacity increases, participants become biased toward inferring that more opaque colors map to larger quantities (opaque-is-more bias). These biases work together on light backgrounds and conflict on dark backgrounds. Under such conflicts, the opaque-is-more bias can negate, or even supersede the dark-is-more bias. The results suggest that if a design goal is to produce colormaps that match people's inferred mappings and are robust to changes in background color, it is beneficial to use colormaps that will not appear to vary in opacity on any background color, and to encode larger quantities in darker colors.

MiningVis: Visual Analytics of the Bitcoin Mining Economy

“…In addition to what information they found useful or missing we asked them about their thoughts on the micro-entries form as the method was just recently proposed for insight evaluation. The survey consisted of a System Usability Scale (SUS) [15] and a variant of the ICE-T questionnaire [65]. Four participants agreed to be interviewed and five filled the survey.…”

“…We evaluated the value of the implemented visualizations with an adaptation of the ICE-T questionnaire [65]. We used a scale of five to score the heuristics (from 1-Strongly Disagree to 5-Strongly Agree) and calculated the average score for each value component.…”

Evaluating Multivariate Network Visualization Techniques Using a Validated Design and Crowdsourcing Approach

“…al. [36] and added a set of custom questions tailored to our design decisions. We ran a pilot with one expert to evaluate our survey, which led to design recommendations we implemented but also to issues identified in our survey.…”

StanceVis Prime: visual analysis of sentiment and stance in social media texts

“…The table in Fig. 3a acts as a legend for the colors associated with each target/domain, and it also enumerates the corresponding data series with their abbreviated titles and sparkline plots (Tufte 2006). The category labels are also used in other parts of the interface to avoid the introduction of complicated glyphs for multiple data series.…”

“…During the session which lasted several hours, we (1) explained the Fig. 11 The document lists for European Politics in case study B: a relevant tweets retrieved for the evening of September 9, 2018; b relevant Reddit comments retrieved for the evening of September 9, 2018; and c relevant Reddit comments retrieved for the morning of September 10, 2018 underlying design choices, (2) demonstrated the visual interface functionality, (3) allowed the user to engage in free data exploration with a think-aloud protocol, and (4) conducted a semi-structured interview using the ICE-T questionnaire by Wall et al (2019) focusing on the value of visualization.…”