Predicting data science sociotechnical execution challenges by categorizing data science projects

Abstract

The challenge in executing a data science project is more than just identifying the best algorithm and tool set to use. Additional sociotechnical challenges include items such as how to define the project goals and how to ensure the project is effectively managed. This paper reports on a set of case studies where researchers were embedded within data science teams and where the researcher observations and analysis was focused on the attributes that can help describe data science projects and the challenges faced by the teams executing these projects, as opposed to the algorithms and technologies that were used to perform the analytics. Based on our case studies, we identified 14 characteristics that can help describe a data science project. We then used these characteristics to create a model that defines two key dimensions of the project. Finally, by clustering the projects within these two dimensions, we identified four types of data science projects, and based on the type of project, we identified some of the sociotechnical challenges that project teams should expect to encounter when executing data science projects.

Demystifying Data Science Projects: A Look on the People and Process of Data Science Today

“…In a further study, Saltz et al [17] could label data science projects with two dimensions (infrastructure and discovery), based on which they could identify four different types of data science projects depending on where projects could be placed on the axes. The project types were: Hard to Justify, Exploratory, Well-Defined and Small data.…”

“…There are only few (e.g. [10,17]) data science interview studies over multiple companies. Kandel et al [10] concentrate on individual analyst skill set and workflow mentioning within team collaboration briefly.…”

Factors that Influence the Selection of a Data Science Process Management Methodology: An Exploratory Study

“…However, the 4 Vs are sometimes not sufficient to describe a project. Hence, we focus on Saltz et al's [37] characterization of data science projects, in which there are two key dimensions. One key attribute is the level of discovery and the other is the level of technical infrastructure required for the project (this second attribute includes, from a project management perspective, the four Vs).…”

Big Data Redux: New Issues and Challenges Moving Forward

“…It described a methodology based on IT value theory and workgroup ideation, guiding big data idea generation, idea assessment and implementation management. Another study [28] Other management challenges discussed [29] are that managers who track business intelligence and analytics costs believe they have already established a higher degree of big data implementations. Another paper [30] discussed big data-driven innovations as key in improving healthcare system sustainability through public-private partnerships (PPPs).…”

“…Several studies [5,18] discussed the big data validation challenges. One main challenge is determining if the data to be analyzed have no issues, such as mismatching or missing values, needing cleansing, or determining if the data meets quality needs [28]. We noted that the acceptable level of data quality depends on the purpose of that analysis.…”