Good systems, bad data?: Interpretations of AI hype and failures

Abstract

Artificial intelligence (AI), including machine learning (ML), is widely viewed as having substantial transformative potential across society, and novel implementations of these technologies promise new modes of living, working, and community engagement. Data and the algorithms that operate upon it thus operate under an expansive ethical valence, bearing consequence to both the development of these potentially transformative technologies and our understanding of how best to manage and support its impact. This paper reports upon an interview‐driven study of stakeholders engaged with technology development, policy, and law relating to AI. Among our participating stakeholders, unexpected outcomes and flawed implementations of AI, especially those leading to negative social consequences, are often attributed to ill‐structured, incomplete, or biased data, and the algorithms and interpretations that might produce negative social consequence are seen as neutrally representing the data, or otherwise blameless in that consequence. We propose a more complex infrastructural view of the tools, data, and operation of AI systems as necessary to the production of social good, and explore how representations of the successes and failures of these systems, even among experts, tend to valorize algorithmic analysis and locate fault at the quality of the data rather than the implementation of systems.

Expert views about missing AI narratives: is there an AI story crisis?

“…Narrative plays an integral part of these visions, and they are shifting all the time (Bory 2019 ). Of course, what stories and narratives have in common is their tendency to be the subject of hype (Blom and Hansen 2015 ; Samuel et al 2021 ; Slota et al 2020 ). Hence, there is not only a requirement for new narratives, but also an increase in public understanding about the need to interrogate narrative features: who is telling the story, what is its genre, and what are their communicative purposes?…”

Tradeoffs all the way down: Ethical abduction as a decision-making process for data-intensive technology development

“…In this body of work, data, data science, algorithms, machine learning, and artificial intelligence are often treated as closely related phenomena (Peters et al, 2020; Saltz et al, 2019; Whittlestone et al, 2019). Data are understood to be the foundation upon which algorithmic technologies are built, and as such are implicated in conversations about the ethics of machine learning and artificial intelligence (Slota et al, 2020). Although the case presented herein does not involve the development of AI, a review of literature relevant to data ethics touches on this broader corpus of scholarship.…”

“…Scholarship on practitioners developing artificial intelligence and computational technologies more generally highlight how individual ethical agency is “extrinsically bound” (Orr and Davis, 2020: 725) by more powerful actors, structures, and cultural forces (Metcalf et al, 2019). At the same time, seemingly “mundane” decisions made in the course of routine work regarding the structure, format, and quality of data have profound downstream ethical implications (Leonelli, 2016)—a fact that is frequently not lost on practitioners, who often make a “scapegoat” of “bad data” when their technologies go awry (Slota et al, 2020: 2). Practitioners can recognize particular moments when they face decisions of ethical consequence through the incorporation of activities that prompt reflection and discussion about ethical values (Shilton, 2013).…”

Locating the work of artificial intelligence ethics

“…This meaning making activity is inherently a social process, and is influenced by the cultural perspectives and values of the relevant stakeholders. While there is an increasingly broad amount of data collected (particularly behavioral and social data), and increasingly an ethos of data interoperability and reuse (Ribes & Bowker, 2009) among scientists, assessment, understanding and selection of data still remains a vital, ill‐understood step in the development of AI and data‐intensive modeling and analysis in all its variety of definitions (Slota, Fleischmann, et al, 2020; Slota, Hoffman, et al, 2020). While there are many efforts to encourage and manage interoperability and data assessment (Gudivada et al, 2017; Janssen et al, 2014; Ribes & Bowker, 2009; Salminen & Pallai, 2007), data collection often takes place outside of the context of modeling or AI work, and a necessary prior step in undertaking this work is in the discovery and assessment of resources—be they data, tools, or even domains where their work might find application (Slota, Fleischmann, et al, 2020; Slota, Hoffman, et al, 2020).…”