Visualizing dimensionally-reduced data

Abstract

We characterize five task sequences related to visualizing dimensionally-reduced data, drawing from data collected from interviews with ten data analysts spanning six application domains, and from our understanding of the technique literature. Our characterization of visualization task sequences for dimensionallyreduced data fills a gap created by the abundance of proposed techniques and tools that combine high-dimensional data analysis, dimensionality reduction, and visualization, and is intended to be used in the design and evaluation of future techniques and tools. We discuss implications for the evaluation of existing work practices, for the design of controlled experiments, and for the analysis of post-deployment field observations.

Constructing and Evaluating Visualisation Task Classifications: Process and Considerations

“…Three of these publications [Mac95,TC05,SM13] were not available to us. We considered [Mun14] and [BSIM14] together, as the discussion relating to the construction and evaluation of this classification is presented in [BSIM14]. The literature was reviewed with regard to (1) terminology employed, (2) the intended uses of classifications, (3) the construction methods employed, (4) the aspects of classifications that were evaluated, and (5) the methods employed in evaluating these aspects.…”

“…Roth et al [Rot13] suggest that the empirically-derived nature of their framework makes it ecologically valid, and therefore offers advantages over other works. Amar et al [AS05] reflect on the limitations of using student questions as the basis of their classification, while Brehemer et al [BSIM14] acknowledge the limitations of using interviews to gather task sequences, noting that their classification may be incomplete due to sampling or observer bias. Wehrend and Lewis [WL90] consider the rigour with which their categories were selected, and whether an abstract mathematical approach would provide a "cleaner" analysis.…”

“…They can be of use when selecting representative tasks for use in experiments, acting as a "checklist" covering the range of possible tasks for inclusion, in a manner similar to that of the design process (Section 4.4). Brehmer et al [BSIM14] outline potential uses of their characterisation of task sequences in four of Lam et al's [LBI * 11] empirical evaluation scenarios: as a lexicon for coding observations when understanding working practices (as described in Section 4.4.2); to inform the design of experimental procedures when evaluating user performance; when specifying tasks for use when evaluating user experience, either as instructions in experiments or when constructing questionnaires and interview questions relating to user experience; and when coding observed behaviour when evaluating visual data analysis and reasoning. The primary use of task classifications for evaluation purposes that we found in our review was their use in characterising systems in terms of task support.…”

“…Schulz [GZ09]. Far less common were the use of empirical methods to elicit tasks (8 of 27 classifications), including interviews with domain experts (either in a single domain [MMP09,Rot13], or across multiple domains [BSIM14]); surveys of those familiar with visualisation [AES05]; observational studies of people using visualisation sys- tems [GZ09, VPF06]; reviews of commercial systems [YKSJ07]; and expert reviews of the resultant classification to find missing tasks [APS14].…”

“…Of these, only five reported the method of categorisation employed when grouping the tasks. These included identifying common characteristics and uses of techniques and abstracting beyond the details of particular implementations [SP13]; an iterative coding process [BSIM14]; affinity diagramming [AES05, YKSJ07]; and card sorting with domain experts [Rot13].…”