Evaluating the Usability of a Tool for Visualizing the Uncertainty of the Future Global Water Balance

Abstract

We describe the development of software that is intended to enable decision makers (and their scientific advisors) to visualize uncertainties associated with the future global water balance. This is an important task because the future water balance is a function of numerous factors that are not precisely known, including the historical climatology, the model of potential evapotranspiration, the soil water holding capacity, and the global circulation models (GCMs) used to predict the effect of increased CO 2 in the atmosphere. In developing the software, we utilized the principles of usability engineering. In our case, we utilized six steps: prototype development, evaluation by domain experts, software revision, evaluation by usability experts, software revision, and evaluation by decision makers. Although this approach led to an improved piece of software, decision makers should have been involved earlier in the software design process, possibly at step two (instead of the domain experts). Decision makers found the notion of uncertainty discomforting, but their positive comments regarding the software suggest that it could prove beneficial, especially with improvements in spatial and temporal resolution. One interesting characteristic of our approach was the utilization of a wall-size display measuring 25 x 6 feet. The wall-size display engendered great interest, but determining whether it is truly effective will require a study that directly compares it with more traditional approaches.

How to Assess Visual Communication of Uncertainty? A Systematic Review of Geospatial Uncertainty Visualisation User Studies

“…Examples include colour hue and colour value (a term under which we subsume value, lightness and brightness) (e.g. Aerts et al, 2003;Edwards and Nelson, 2001;Leitner and Buttenfield, 2000;MacEachren et al, 1998;Nadav-Greenberg et al, 2008;Retchless, 2012;Schweizer and Goodchild, 1992;Slocum et al, 2003), transparency (e.g. Drecki, 2002;Newman and Lee, 2004;Slocum et al, 2003;Viard et al, 2011), or colour saturation that is used in a number of studies (e.g.…”

“…Aerts et al, 2003;Edwards and Nelson, 2001;Leitner and Buttenfield, 2000;MacEachren et al, 1998;Nadav-Greenberg et al, 2008;Retchless, 2012;Schweizer and Goodchild, 1992;Slocum et al, 2003), transparency (e.g. Drecki, 2002;Newman and Lee, 2004;Slocum et al, 2003;Viard et al, 2011), or colour saturation that is used in a number of studies (e.g. Drecki, 2002;Kubíček and Šašinka, 2011;Kunz et al, 2011;Leitner and Buttenfield, 2000;Retchless, 2012;Sanyal et al, 2009).…”

“…when additional graphical objects are used to represent uncertainty, typically approaches using glyphs or error bars (Alberti, 2013;Drecki, 2002;Sanyal et al, 2009;Slocum et al, 2003), grid-based techniques (Kardos et al, 2007;Kinkeldey et al, 2014) or contouring (Senaratne et al, 2012).…”

“…As the display typically can become complex when uncertainty is added to data depictions, there have also been numerous attempts to utilize dynamic views. Some of these use non-interactive animation (Aerts et al, 2003;Blenkinsop et al, 2000;Evans, 1997;Kardos et al, 2003;Kardos et al, 2007;Zhang et al, 2008) and some incorporate interactive interfaces (Alberti, 2013;Blenkinsop et al, 2000;Evans, 1997;Gerharz and Pebesma, 2009;Slocum et al, 2003;Senaratne et al, 2012).…”

“…A number of studies define groups with different levels of expertise (e.g. novices versus experts) to assess its impact on the results (Evans, 1997;Kubíček and Šašinka, 2011;Nadav-Greenberg et al, 2008;Schweizer and Goodchild, 1992;Slocum et al, 2003). All in all, the type and level of expertise of participants often remain unclear and groups are usually heterogeneous in expertise.…”