Algorithmic Impact Assessments and Accountability

Abstract

Algorithmic impact assessments (AIAs) are an emergent form of accountability for organizations that build and deploy automated decision-support systems. They are modeled after impact assessments in other domains. Our study of the history of impact assessments shows that "impacts" are an evaluative construct that enable actors to identify and ameliorate harms experienced because of a policy decision or system. Every domain has different expectations and norms around what constitutes impacts and harms, how potential harms are rendered as impacts of a particular undertaking, who is responsible for conducting such assessments, and who has the authority to act on them to demand changes to that undertaking. By examining proposals for AIAs in relation to other domains, we find that there is a distinct risk of constructing algorithmic impacts as organizationally understandable metrics that are nonetheless inappropriately distant from the harms experienced by people, and which fall short of building the relationships required for effective accountability. As impact assessments become a commonplace process for evaluating harms, the FAccT community, in its efforts to address this challenge, should A) understand impacts as objects that are co-constructed accountability relationships, B) attempt to construct impacts as close as possible to actual harms, and C) recognize that accountability governance requires the input of various types of expertise and affected communities. We conclude with lessons for assembling cross-expertise consensus for the co-construction of impacts and building robust accountability relationships.

From human resources to human rights: Impact assessments for hiring algorithms

“…Depending on the context in which the HRIA is conducted, using the HRIA as a standalone impact assessment may not always be enough to address the myriad risks and untold effects that algorithms may have. In a recent paper, Metcalf et al (2021) cite how difficult it is to ensure that the kinds of impacts identified within the scope of AIAs are accurately related to actual or potential harms to those most likely to experience them. As they point out, "What counts as an adequate assessment, when that assessment happens, and how stakeholders are made accountable to each other are contested outcomes shaped by fraught power relationships" (Metcalf et al,p.…”

Governing Algorithmic Systems with Impact Assessments: Six Observations

“…Alongside the actors and the forum to establish accountability, impact assessments share a number of other constitutive components outlined in Table 1 (Moss et al 2021). While details vary, these components appear consistently across domains.…”

“…As we argue below, and in greater depth elsewhere (Moss et al 2021), the accountability relationship between an actor who builds and designs systems and a forum capable of demanding changes is central to operationalizing an impact assessment accountability-act-of-2019/; Selbst, Andrew, Madeleine Clare Elish and Mark Latonero. "Accountable Algorithmic Futures."…”

Artificial Intelligence Project Success Factors: Moral Decision-Making with Algorithms

“…Interpretable models refer to having a model design that is reliable, understandable, and possible for expert users to explain the predictions [27,37]. Choices allow users to decide what to do with model results or, in other words, provides a degree of human control [23,27,37,49,58].…”