Constructing and Evaluating Visualisation Task Classifications: Process and Considerations

Abstract

Categorising tasks is a common pursuit in the visualisation research community, with a wide variety of taxonomies, typologies, design spaces, and frameworks having been developed over the last three decades. While these classifications are universally purported to be useful in both the design and evaluation processes and in guiding future research, remarkably little attention has been paid to how these frameworks have-and can be-constructed and evaluated. In this paper we review the task classification literature and report on current practices in construction and evaluation. We consider the stages of task classification construction and identify the associated threats to validity arising at each stage and in response to the different methods employed. We provide guidance on suitable validation approaches in order to mitigate these threats. We also consider the appropriateness of evaluation strategies according to the different aspects of the classification which they evaluate. In so doing, we seek to provide guidance for developers of classifications in determining appropriate construction and evaluation strategies when developing a classification, and also for those selecting between competing classifications for use in the design and evaluation processes.

The State of the Art in Multilayer Network Visualization

“…Task taxonomies are widely accepted to be useful for the evaluation process [KK17] and the tasks described in Section 4.1 should support the evaluation of multilayer visualization systems and techniques. As described in Section 4.6, there is a lack of empirical evaluation for multilayer network visualizations.…”

From Taxonomy to Requirements: A Task Space Partitioning Approach

“…The use of facilitated workshops, involving a group of users with the aim of developing user stories, is an extremely popular approach for agile projects. While user stories are effective in capturing the essence of functional requirements, they share the same threats to validity that have been reported for qualitative techniques [12]. Namely, structuring methods need to be employed to draw out the detail of the functionality or to provide a coherent view of the, often numerous, user stories.…”

“…They provide structure to these decisions. Specific to the requirements analysis phase are several task taxonomies that have been introduced in the visualization literature [1,3,4,6,12,19,25,30]. Amar and Stasko [3] and Sediq and Parsons [26] advocate the use of these classifications as a systematic basis for thinking about the design process.…”

“…Recently, Kerracher et al [12] promoted the usefulness of classification for task understanding, data abstraction and technique design. They argued that by setting out the range of potential tasks of interest (i.e.…”

“…Bertin defined three levels for this dimension (with an additional intermediate level for subsets as a whole) [5]. Recent taxonomies by Kerracher et al [12,13] found this three-level approach to be useful within specific visualization contexts. For example, for network visualization, it was found useful to separate tasks that require the analysis of clusters or groupings of nodes versus those that targeted the network as a whole.…”