Smallset Timelines: A Visual Representation of Data Preprocessing Decisions

Abstract

Data preprocessing is a crucial stage in the data analysis pipeline, with both technical and social aspects to consider. Yet, the attention it receives is often lacking in research practice and dissemination. We present the Smallset Timeline, a visualisation to help reflect on and communicate data preprocessing decisions. A "Smallset" is a small selection of rows from the original dataset containing instances of dataset alterations. The Timeline is comprised of Smallset snapshots representing different points in the preprocessing stage and captions to describe the alterations visualised at each point. Edits, additions, and deletions to the dataset are highlighted with colour. We develop the R software package, smallsets, that can create Smallset Timelines from R and Python data preprocessing scripts. Constructing the figure asks practitioners to reflect on and revise decisions as necessary, while sharing it aims to make the process accessible to a diverse range of audiences. We present two case studies to illustrate use of the Smallset Timeline for visualising preprocessing decisions. Case studies include software defect data and income survey benchmark data, in which preprocessing affects levels of data loss and group fairness in prediction tasks, respectively. We envision Smallset Timelines as a go-to data provenance tool, enabling better documentation and communication of preprocessing tasks at large. CCS CONCEPTS• Human-centered computing → Visualization toolkits.

Leveraging Large Language Models to Enhance Domain Expert Inclusion in Data Science Workflows

“…To bridge the expertise gap, systems should be designed to aid domain experts in understanding data science techniques in a manner that better matches their mental model of data [21]. Visualizations of data changes [17,24,32] have proved to be an effective means of supporting understanding beyond serving the function of documentation. Recent work in Explainable AI [7,9,11,18,50] has also demonstrated ways to make ML components more interpretable to non-experts through visualization and direct manipulation interactions [2,29,39].…”

“…Informed by the challenges and potential solutions identified in prior research, we explore the use of LLMs to supplement human effort in translating code into natural language and explainable visualizations. In CellSync, we build on the Smallset Timelines visualization technique [24] to select and visualize a digestible subset of rows and columns for domain experts. Further, we utilize LLMs' code summarization capabilities [1] and refine these summaries through targeted prompt engineering to tailor them towards domain experts.…”

“…Both the extension and the dashboard provide a chat feature for domain experts (Figure 1d) and data scientists (Figure 2) to exchange comments. Each SnapGrid, inspired by Smallset Timelines [24] which visualizes data changes in grid-based snapshots, shows a static 9-by-9 subset of the data; the subset is selected to maximize the coverage of the changes tied to the data operation. On each square that represents a cell in the dataframe, the cell's previous and new values are displayed with an arrow between them.…”

“…4.1.2 Computing Subset for SnapGrid. Building upon the Smallset Timelines subset computation [24], we devise an algorithm to select a subset of 9 rows by 9 columns that maximizes the coverage of value changes for SnapGrid. The algorithm selects the dataset rows with the highest number of changed values, and the columns that have been directly affected or are implicitly involved in column changes (e.g.…”

A Unified Framework for Specification Tests of Continuous Treatment Effect Models

“…Standardised description of specific preprocessing steps is challenging due to the wide variety of possible data alterations. Moreover, as observed by Lucchesi et al (2022), definitions of data preprocessing vary with audience and context from highly specific lists of tasks, to broadly encompassing boundaries within a longer data pipeline. Existing provenance tools such as (Lucchesi et al 2022;Kai Xiong et al 2022;Wang et al 2022) attempt to achieve generality by comparing dataset snapshots at various points in a preprocessing pipeline.…”