Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning

Cooper, A. Feder; Laufer, Benjamin; Moss, Emanuel; Nissenbaum, Helen

doi:10.1145/3531146.3533150

Cited by 45 publications

(26 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With increased awareness of the need to anticipate harms early in product development [172], designers and researchers are central actors in pursuing harm reduction [34,49,87]. Anticipating harms requires considering how technological affordances shape their use and impact [76,175].…”

Section: Identifying and Anticipating Harms In Practicementioning

confidence: 99%

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Shelby¹,

Rismani²,

Henne³

et al. 2022

Preprint

View full text Add to dashboard Cite

Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems arising at the micro-, meso-, and macro-levels of society. We present an applied taxonomy of sociotechnical harms to support more systematic surfacing of potential harms in algorithmic systems. Based on a scoping review of computing research (n=172), we identified five major themes related to sociotechnical harms -representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms -and sub-themes. We describe these categories and conclude with a discussion of challenges and opportunities for future research.

show abstract

Section: Identifying and Anticipating Harms In Practicementioning

confidence: 99%

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Shelby¹,

Rismani²,

Henne³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To borrow a phrase from Jack Balkin, such "code is lawless" [1, p. 52], since the unpredictability that results from non-determinism presents key problems for thinking of code as being constitutive of law. 15…”

Section: Non-deterministic Code Is Lawlessmentioning

confidence: 99%

“…In ML systems, we can have full access to both the inputs and subsequent outputs, while having no clear understanding of how the mapping from one to the other occurred. In other words, unlike the law, ML functions defy explanation and reasonable justification, which in turn raises fundamental questions about the legitimacy of using ML as a decision-making tool and muddies the ability to determine accountability when these tools cause harms [15]. In short, ML's problem with "explainability" shows how the analogy essentially and inescapably falls short; both the law and ML may behave like functions, but functions that are fundamentally different in kind.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Non-Determinism and the Lawlessness of Machine Learning Code

Cooper,

Frankle,

De Sa

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Legal literature on machine learning (ML) tends to focus on harms, and as a result tends to reason about individual model outcomes and summary error rates. This focus on model-level outcomes and errors has masked important aspects of ML that are rooted in its inherent non-determinism. We show that the effects of non-determinism, and consequently its implications for the law, instead become clearer from the perspective of reasoning about ML outputs as probability distributions over possible outcomes. This distributional viewpoint accounts for non-determinism by emphasizing the possible outcomes of ML. Importantly, this type of reasoning is not exclusive with current legal reasoning; it complements (and in fact can strengthen) analyses concerning individual, concrete outcomes for specific automated decisions. By clarifying the important role of non-determinism, we demonstrate that ML code falls outside of the cyberlaw frame of treating "code as law, " as this frame assumes that code is deterministic. We conclude with a brief discussion of what work ML can do to constrain the potentially harm-inducing effects of non-determinism, and we clarify where the law must do work to bridge the gap between its current individual-outcome focus and the distributional approach that we recommend.

show abstract

“…In addition to these regulatory discussions, academic researchers surface the impact of algorithmic systems in public sector and call for algorithmic accountability (Barocas and Selbst, 2016;Calo and Citron 2021;Cooper et al 2022;Crump, 2016;Diakopoulos, 2014;Eubanks, 2018;Kroll et al 2017;O'Neil 2016;Pasquale, 2015;Richardson et al 2019;Schwartz, 1992;Veale et al 2018;Young et al 2019), and impact assessments to be mandatory (A Civil Society Statement 2021; Ada Lovelace Institute 2021; Kaminski and Malgieri 2019;Reisman et al 2018). A robust literature identifies the need for transparency and public disclosures.…”

mentioning

confidence: 99%

Public procurement of artificial intelligence systems: new risks and future proofing

Hickok

2022

AI & Soc

View full text Add to dashboard Cite

Public entities around the world are increasingly deploying artificial intelligence (AI) and algorithmic decision-making systems to provide public services or to use their enforcement powers. The rationale for the public sector to use these systems is similar to private sector: increase efficiency and speed of transactions and lower the costs. However, public entities are first and foremost established to meet the needs of the members of society and protect the safety, fundamental rights, and wellbeing of those they serve. Currently AI systems are deployed by the public sector at various administrative levels without robust due diligence, monitoring, or transparency. This paper critically maps out the challenges in procurement of AI systems by public entities and the long-term implications necessitating AI-specific procurement guidelines and processes. This dual-prong exploration includes the new complexities and risks introduced by AI systems, and the institutional capabilities impacting the decision-making process. AI-specific public procurement guidelines are urgently needed to protect fundamental rights and due process.

show abstract

Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning

Cited by 45 publications

References 49 publications

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Non-Determinism and the Lawlessness of Machine Learning Code

Public procurement of artificial intelligence systems: new risks and future proofing

Contact Info

Product

Resources

About