A User Study to Compare Four Uncertainty Visualization Methods for 1D and 2D Datasets

Abstract

Many techniques have been proposed to show uncertainty in data visualizations. However, very little is known about their effectiveness in conveying meaningful information. In this paper, we present a user study that evaluates the perception of uncertainty amongst four of the most commonly used techniques for visualizing uncertainty in one-dimensional and twodimensional data. The techniques evaluated are traditional errorbars, scaled size of glyphs, color-mapping on glyphs, and colormapping of uncertainty on the data surface. The study uses generated data that was designed to represent the systematic and random uncertainty components. Twenty-seven users performed two types of search tasks and two types of counting tasks on 1D and 2D datasets. The search tasks involved finding data points that were least or most uncertain. The counting tasks involved counting data features or uncertainty features. A 4 4 full-factorial ANOVA indicated a significant interaction between the techniques used and the type of tasks assigned for both datasets indicating that differences in performance between the four techniques depended on the type of task performed. Several one-way ANOVAs were computed to explore the simple main effects. Bonferronnis correction was used to control for the family-wise error rate for alpha-inflation. Although we did not find a consistent order among the four techniques for all the tasks, there are several findings from the study that we think are useful for uncertainty visualization design. We found a significant difference in user performance between searching for locations of high and searching for locations of low uncertainty. Errorbars consistently underperformed throughout the experiment. Scaling the size of glyphs and color-mapping of the surface performed reasonably well. The efficiency of most of these techniques were highly dependent on the tasks performed. We believe that these findings can be used in future uncertainty visualization design. In addition, the framework developed in this user study presents a structured approach to evaluate uncertainty visualization techniques, as well as provides a basis for future research in uncertainty visualization.

How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies

“…Drecki, 2002;Newman and Lee, 2004;Slocum et al, 2003;Viard et al, 2011), or colour saturation that is used in a number of studies (e.g. Drecki, 2002;Kubíček and Šašinka, 2011;Kunz et al, 2011;Leitner and Buttenfield, 2000;Retchless, 2012;Sanyal et al, 2009).…”

“…when additional graphical objects are used to represent uncertainty, typically approaches using glyphs or error bars (Alberti, 2013;Drecki, 2002;Sanyal et al, 2009;Slocum et al, 2003), grid-based techniques (Kardos et al, 2007;Kinkeldey et al, 2014) or contouring (Senaratne et al, 2012).…”

“…Few authors explicitly justify the choice of tasks, e.g. Sanyal et al (2009) asked domain experts which tasks they would find important for their application. For this reason, they chose the search of hotspots in uncertainty ('[the expert said he] would be interested in looking at regions of extreme (high or low) uncertainty', p. 1213) and the count of features of a specific combination of data and uncertainty ('He also wanted to be able to discern features in the data, in the presence of uncertainty', p. 1213).…”

“…extremely high or low values. Several studies incorporate such tasks, for example, the search for the highest data value (Viard et al, 2011), the lowest and/or the highest uncertainty (Sanyal et al, 2009;Wray, 2007), or the identification of patterns such as clusters in the data (Edwards and Nelson, 2001;MacEachren et al, 1998) or in uncertainty only (Drecki, 2002;Edwards and Nelson, 2001;Sanyal et al, 2009). Sanyal et al (2009) extend this task type further to multiple entities using a counting task for data values and uncertainty meaning that the number of clusters in data and uncertainty has to be determined.…”

“…Visualisation of uncertainty has attracted substantial attention over more than two decades. Much of the work has focused on developing typologies of uncertainty that represent various aspects of data and how it might be signified (Buttenfield and Weibel, 1988;Pang et al, 1997;Sanyal et al, 2009;Thomson et al, 2005) and on developing methods to depict uncertainty visually (e.g. Cedilnik and Rheingans, 2000;Ehlschlaeger et al, 1997;Sanyal et al, 2010;Wittenbrink et al, 1996).…”