Orienting, Framing, Bridging, Magic, and Counseling: How Data Scientists Navigate the Outer Loop of Client Collaborations in Industry and Academia

Abstract

Data scientists often collaborate with clients to analyze data to meet a client's needs. What does the end-to-end workflow of a data scientist's collaboration with clients look like throughout the lifetime of a project? To investigate this question, we interviewed ten data scientists (5 female, 4 male, 1 non-binary) in diverse roles across industry and academia. We discovered that they work with clients in a six-stage outer-loop workflow, which involves 1) laying groundwork by building trust before a project begins, 2) orienting to the constraints of the client's environment, 3) collaboratively framing the problem, 4) bridging the gap between data science and domain expertise, 5) the inner loop of technical data analysis work, 6) counseling to help clients emotionally cope with analysis results. This novel outer-loop workflow contributes to CSCW by expanding the notion of what collaboration means in data science beyond the widely-known inner-loop technical workflow stages of acquiring, cleaning, analyzing, modeling, and visualizing data. We conclude by discussing the implications of our findings for data science education, parallels to design work, and unmet needs for tool development.CCS Concepts: • Human-centered computing → Human computer interaction (HCI).

Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook

“…Researchers report that practitioners without a background in AI (e.g., designers, product managers, domain experts) have challenges in engaging data and AI [24,78]. Product professionals struggle to understand what AI can do, and cannot easily formulate business problems into data science problems [54,65,72,77,78,108]. From an interaction design perspective, AI presents unique challenges for ideating and rapid prototyping novel, implementable AI experiences [24,90,104] (see [104] for a review of AI's design challenges).…”

“…In addition to challenges specific for individual roles, there are many challenges in cross-disciplinary collaboration, mainly due to a lack of shared workflow, shared language, and shared expertise [31,50,78]. Some researchers characterized this as a gap between AI and product expertise [102,105]: data science and AI teams struggle to elicit user requirements from product teams, and they tend to overlook how the AI system will generate value for users [54,72,78]. Product teams (e.g., PMs, designers) tend to envision AI concepts that cannot be built or too high stakes to deploy an imperfect AI model [24,104].…”

“…A small but growing number of studies explored the emergent best practices for designing AI products. Successful practitioners formed close collaborations between data science and product teams [54,90,103,105], where product teams worked to sensitize data science and AI partners to human-centered approaches [107]. Design teams in particular served as facilitators on AI teams to establish a shared understanding and alignment between different disciplines using interim artifacts and visualizations [105,107].…”

“…Another study found that design teams created internal AI resources, such as an AI capability matrix and a collection of examples, for educating themselves and engaging their clients in early phase ideation and problem formulation [105]. Similarly, studies on data science and AI practitioners revealed that they worked to sensitize their cross-functional partners, such as PMs and domain experts, to AI concepts [54,72,78]. The way they worked resembled design practice [54]; they probed their collaborators to get at the underlying problem, to frame and reframe it as a well-defined data science problem [54,72,76].…”

“…Similarly, studies on data science and AI practitioners revealed that they worked to sensitize their cross-functional partners, such as PMs and domain experts, to AI concepts [54,72,78]. The way they worked resembled design practice [54]; they probed their collaborators to get at the underlying problem, to frame and reframe it as a well-defined data science problem [54,72,76]. Finally, research showed that effective AI teams had an approach similar to startups [83], where they would focus on a minimum viable product at the intersection of what is feasible with AI models and data, and what is desirable and viable for customers and users [96,103,105,107].…”