The Algorithmic Imprint

Abstract

When algorithmic harms emerge, a reasonable response is to stop using the algorithm to resolve concerns related to fairness, accountability, transparency, and ethics (FATE). However, just because an algorithm is removed does not imply its FATE-related issues cease to exist. In this paper, we introduce the notion of the "algorithmic imprint" to illustrate how merely removing an algorithm does not necessarily undo or mitigate its consequences. We operationalize this concept and its implications through the 2020 events surrounding the algorithmic grading of the General Certificate of Education (GCE) Advanced (A) Level exams, an internationally recognized UKbased high school diploma exam administered in over 160 countries. While the algorithmic standardization was ultimately removed due to global protests, we show how the removal failed to undo the algorithmic imprint on the sociotechnical infrastructures that shape students', teachers', and parents' lives. These events provide a rare chance to analyze the state of the world both with and without algorithmic mediation. We situate our case study in Bangladesh to illustrate how algorithms made in the Global North disproportionately impact stakeholders in the Global South. Chronicling more than a year-long community engagement consisting of 47 interviews, we present the first coherent timeline of "what" happened in Bangladesh, contextualizing "why" and "how" they happened through the lenses of the algorithmic imprint and situated algorithmic fairness. Analyzing these events, we highlight how the contours of the algorithmic imprints can be inferred at the infrastructural, social, and individual levels. We share conceptual and practical implications around how imprint-awareness can (a) broaden the boundaries of how we think about algorithmic impact, (b) inform how we design algorithms, and (c) guide us in AI governance. The imprint-aware design mindset can make the algorithmic development process more human-centered and sociotechnically-informed. CCS CONCEPTS• Human-centered computing → Collaborative and social computing.

Performative Ethics From Within the Ivory Tower: How CS Practitioners Uphold Systems of Oppression

“…To deconstruct systems of oppression that are reified through algorithmic oppression, we argue that socio-technical researchers must have a common foundational base of sociology, ethics, and self-reflexivity. A prevalent limitation in AI ethics work is the avoidance, or inability, to explicitly state the critical structures that shape the world and examine the impacts of sociotechnical systems on society (Ehsan et al, 2022a). Without systemic analysis, we claim work dedicated to positively improving the impact of technology on society will be performative at best and reify systems of oppression at worst.…”

“…Fairness and bias are common words used when considering technology's impact and repairing its negative lasting effects (Ehsan, Singh, Metcalf, & Riedl, 2022a). Other researchers have commented on the various definitions of fairness, such as Hampton's reference to over 21 definitions presented in (Tal, Batsuren, Bogina, Giunchiglia, Hartman, Loizou, Kuflik, & Otterbacher, 2019a).…”

“…The result is explicitly discriminatory models that impact real people, and the harms that have passed can never be undone. The algorithmic imprint (Ehsan, Singh, Metcalf, & Riedl, 2022b) will be longstanding, and until a critical, ethical intervention is presented, these algorithms should be removed. There are very few AI researchers (Hampton, 2021) who advocate for the removal of harmful algorithms.…”

Identifying Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

“…Here, privacy violations may reflect more traditional conceptualizations of privacy attacks or security violations [69,90] and privacy elements beyond what may be protected by regulations or under the traditional purview of a privacy officer [119,159]. For instance, privacy violence may arise from algorithmic systems making predictive inference beyond what users openly disclose [196] or when data collected and algorithmic inferences made about people in one context is applied to another without the person's knowledge or consent through big data flows [119], even after those datasets or systems have been deprecated [50,72]. Even if those inferences are false (e.g., the incorrect assessment of one's sexuality), people or systems can act on that information in ways that lead to discrimination and harm [208].…”