Subtle Bugs Everywhere: Generating Documentation for Data Wrangling Code

Yang, Chenyang; Zhou, Shurui; Guo, Jin; Kästner, Christian

doi:10.1109/ase51524.2021.9678520

Cited by 16 publications

(9 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the release of Jupyter Lab 3.0 introduced a visual debugger that can be used to step through code or to check the value of a variable [20]. This need for more tool support is suggested by other studies as well [5,27].…”

Section: Discussionmentioning

confidence: 99%

“…These studies agree that notebook code is frequently low-quality and error-prone. Closely related to our work is that from Yang et al [27]. They report on a tool to support bug detection in Kaggle notebooks, which they characterized as 'data wrangling code'.…”

Section: Introductionmentioning

confidence: 88%

“…We have many insights about the process of debugging software systems and programs in general [2,12,25], but we lack similar insights for computational notebooks. To the best of our knowledge, there is only one study looking at how computational notebooks are debugged, from Yang et al [27], and it focused on a tool for improving code understanding with program synthesis but did not synthesize strategies used in debugging. Without knowing how computational notebooks are debugged, it is difficult to support-whether with tools or processes-the debugging of computational notebooks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Error Identification Strategies for Python Jupyter Notebooks

Robinson,

Ernst,

Vargas

et al. 2022

Preprint

View full text Add to dashboard Cite

Computational notebooks-such as Jupyter or Colab-combine text and data analysis code. They have become ubiquitous in the world of data science and exploratory data analysis. Since these notebooks present a different programming paradigm than conventional IDE-driven programming, it is plausible that debugging in computational notebooks might also be different. More specifically, since creating notebooks blends domain knowledge, statistical analysis, and programming, the ways in which notebook users find and fix errors in these different forms might be different. In this paper, we present an exploratory, observational study into how notebook users find and understand potential errors in notebooks. We presented users with notebooks pre-populated with common notebook errors-errors rooted in either the statistical data analysis, the knowledge of domain concepts, or in the programming. We then analyzed the strategies our study participants used to find these errors and determined how successful each strategy was at identifying errors. Our findings indicate that while the notebook programming environment is different from the environments used for traditional programming, debugging strategies remain quite similar. It is our hope that the insights presented in this paper will help both notebook tool designers and educators make changes to improve how data scientists discover errors more easily in the notebooks they write.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 88%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Error Identification Strategies for Python Jupyter Notebooks

Robinson,

Ernst,

Vargas

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…According to a recent study [63], data handling code tends to be error-prone and often contains subtle issues. We claim that such poor data handling practices will likely cause data smells.…”

Section: Data Handlingmentioning

confidence: 99%

“…As the conversion input represents just a date, developers may be unaware that without further declaration, a time suffix (i.e., "00:00:00") is added. Such issues often go undetected because of the common programmatic style of method chaining when processing data [63]. By sequencing multiple data operations (i.e., method chaining), developers cannot see the intermediate processing results and thus identify problems introduced in the data [61].…”

Section: Data Handlingmentioning

confidence: 99%

Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems

Foidl¹,

Felderer²,

Ramler³

2022

Preprint

View full text Add to dashboard Cite

High data quality is fundamental for today's AI-based systems. However, although data quality has been an object of research for decades, there is a clear lack of research on potential data quality issues (e.g., ambiguous, extraneous values). These kinds of issues are latent in nature and thus often not obvious. Nevertheless, they can be associated with an increased risk of future problems in AI-based systems (e.g., technical debt, data-induced faults). As a counterpart to code smells in software engineering, we refer to such issues as Data Smells. This article conceptualizes data smells and elaborates on their causes, consequences, detection, and use in the context of AI-based systems. In addition, a catalogue of 36 data smells divided into three categories (i.e., Believability Smells, Understandability Smells, Consistency Smells) is presented. Moreover, the article outlines tool support for detecting data smells and presents the result of an initial smell detection on more than 240 real-world datasets.

show abstract