How Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork

Abstract

Domain experts play an essential role in data science by helping data scientists situate their technical work beyond the statistical analysis of large datasets. How domain experts themselves may engage with data science tools as a type of end-user remains largely invisible. Understanding data science as domain expert-driven depends on understanding how domain experts use data. Drawing on an ethnographic study of a craft brewery in Korea, we show how craft brewers worked with data by situating otherwise abstract data within their brewing practices and settings. We contribute theoretical insight into how domain experts use data distinctly from technical data scientists in terms of their view of data (situated vs. abstract), purposes for engaging with data (guiding processes over predicting outcomes), and overall goals of using data (flexible control vs. precision). We propose four ways in which working with data can be supported through the design of data science tools, and discuss how craftwork can be a useful lens for integrating domain expert-driven understandings of data science into CSCW and HCI research.

Leveraging Large Language Models to Enhance Domain Expert Inclusion in Data Science Workflows

“…Data scientists often spend large amounts of time preparing concrete data examples to communicate with domain experts [37], and meaningful ML stakeholder participation requires scaffolds to aid the exchange of technical and non-technical considerations [45]. To bridge the expertise gap, systems should be designed to aid domain experts in understanding data science techniques in a manner that better matches their mental model of data [21]. Visualizations of data changes [17,24,32] have proved to be an effective means of supporting understanding beyond serving the function of documentation.…”

Here’s what you need to know about my data: Exploring Expert Knowledge’s Role in Data Analysis.

“…Reflecting on the practices of analysts across different fields, our work also presents findings that the communication direction patterns heavily depend on the role, with an emphasis on the communication of domain expert knowledge. Jung et al presented an in-depth study into EXPLORING EXPERT KNOWLEDGE'S ROLE IN DATA ANALYSIS; 2023 how domain experts work with data [17] and found that they put more value on their data being actionable than the data having abstract qualities, such as high precision. They also discussed conversations with the data-procedures of working with data directly to better understand it [18]-as a critical part of the analysis process, similar to previous works [5,19].…”

Designing Data Science Software for Social Care Organisations

“…Human-AI Collaboration. A growing body of empirical work explores how situated work practices shape and are shaped by the use of data science, and in particular machine learning-enabled, software [44,100]. These studies characterise the appropriation of data science software as a productive process that re-configures power relationships and precipitates new forms of work and expertise [41,42,45,53].…”