Implicit Error, Uncertainty and Confidence in Visualization: An Archaeological Case Study

Abstract

While we know that the visualization of quantifiable uncertainty impacts the confidence in insights, little is known about whether the same is true for uncertainty that originates from aspects so inherent to the data that they can only be accounted for qualitatively. Being embedded within an archaeological project, we realized how assessing such qualitative uncertainty is crucial in gaining a holistic and accurate understanding of regional spatio-temporal patterns of human settlements over millennia. We therefore investigated the impact of visualizing qualitative implicit errors on the sense-making process via a probe that deliberately represented three distinct implicit errors, i.e. differing collection methods, subjectivity of data interpretations and assumptions on temporal continuity. By analyzing the interactions of 14 archaeologists with different levels of domain expertise, we discovered that novices became more actively aware of typically overlooked data issues and domain experts became more confident of the visualization itself. We observed how participants quoted social factors to alleviate some uncertainty, while in order to minimize it they requested additional contextual breadth or depth of the data. While our visualization did not alleviate all uncertainty, we recognized how it sparked reflective meta-insights regarding methodological directions of the data. We believe our findings inform future visualizations on how to handle the complexity of implicit errors for a range of user typologies and for highly data-critical application domains such as the digital humanities.

Communicating Uncertainty in Digital Humanities Visualization Research

“…Underlying implicit errors, i.e. non-recorded measurement errors inherent to datasets, may challenge the confidence domain experts have in data insights [90,98]. Uncertainty can also originate at the visualization stage as visualization mirages, i.e.…”

“…Nevertheless, data uncertainties originating from issues such as implicit errors [98], bias and precision [17] bring skepticism, even mistrust to humanists using visual analysis tools. Datasets in the humanities are often based on historical, incomplete sources and are the result of meticulously intense, manual and subjective collection processes [39,124].…”

Undoing Seamlessness: Exploring Seams for Critical Visualization

“…Uncertainty visualization as a dedicated research area focuses on representing (statistically) uncertain, unclear, or ambiguous aspects of data and data analysis. Displaying statistical [35,41,42] or qualitative uncertainties [5,60,79] allows for a critical reading of data through surfacing some limitations of the data and offering visual cues that direct the interpretation of a visualization. This way, the otherwise seamless representation of data is interrupted and the gap between what the data show and how they should be interpreted is addressed.…”

Data Hunches: Incorporating Personal Knowledge into Visualizations

“…Other approaches use visual approaches to communicate qualitative uncertainty, such as the use of perceptually imprecise visual encoding channels like sketchiness [7] or glyphs [51]. A different approach taken in both the visualization and machine learning communities is to explicitly expose information about the data collection process, providing analysts with contextual information that allows them to incorporate personal knowledge about potential shortcomings of the data during their interpretations [2,26,54].…”

“…Just like in this example, we frequently find that interactions with a visualization tool trigger experts to express knowledge about problems with data, but that these tools leave few options for them to communicate that knowledge. This knowledge is personal and not available to others, a phenomenon reported in other design studies [47,51,54]. However, what if experts could record their hunches directly in a visualization tool in a way that allowed others to interpret the data alongside their hunches?…”