Managing Messes in Computational Notebooks

Abstract

Data analysts use computational notebooks to write code for analyzing and visualizing data. Notebooks help analysts iteratively write analysis code by letting them interleave code with output, and selectively execute cells. However, as analysis progresses, analysts leave behind old code and outputs, and overwrite important code, producing cluttered and inconsistent notebooks. This paper introduces code gathering tools, extensions to computational notebooks that help analysts fnd, clean, recover, and compare versions of code in cluttered, inconsistent notebooks. The tools archive all versions of code outputs, allowing analysts to review these versions and recover the subsets of code that produced them. These subsets can serve as succinct summaries of analysis activity or starting points for new analyses. In a qualitative usability study, 12 professional analysts found the tools useful for cleaning notebooks and writing analysis code, and discovered new ways to use them, like generating personal documentation and lightweight versioning. CCS CONCEPTS • Human-centered computing → Interactive systems and tools; • Software and its engineering → Development frameworks and environments.

Understanding the Role of Alternatives in Data Analysis Practices

“…Woodbury et al [44] define alternatives as "structurally different solutions to a design," while variations are "design solutions with identical model structure, but having different values assigned to parameters." Recent studies on data scientists' work practices have found that they write alternative versions of exploratory code [2,16,24,36] or use multiple alternative models, typically generated by different classes of 1 machine-learning algorithms [18]. Even within a single model, alternatives can also exist.…”

“…Focusing on visual analytics systems, Chen [7] and Guenther [12] describe other types of alternatives, including data and visualization alternatives. Whereas many of those studies observe alternatives in specific contexts, such as design-space exploration and computational notebooks [16,24], our aim is to study the variety of alternatives that might exist in the wild and the processes around those alternatives.…”

“…To support data scientists, Kery et al's Variolite [24] and Head et al's Code Gathering [16] extend computational notebooks and attempt to generalize interactions for dealing with alternatives and histories in exploratory programming tasks. In Variolite, users create alternative code blocks of any size, test different combinations of alternatives, and navigate through them using revision trees.…”

“…Past work has distinguished between variations and versions (e. g., [13,16,24,36]), with these definitions depending on the iteration of the alternative. While this distinction is useful, we find it incomplete-alternatives can evolve throughout the analytic process.…”

Bacatá: Notebooks for DSLs, Almost for Free

“…Currently, there are several studies tackling usability, cognitive, and reproducibility aspects of computational notebooks. For instance, in reproducibility [36,47,48,58], education [44], exploratory programming [2,18,29,30], documentation [76], and data journalism [78]. However, no attention has been paid from the language engineering point of view.…”