AI in the Government: Responses to Failures

Abstract

Artificial Intelligence (AI) is pervading the government and transforming how public services are provided to consumers—from allocation of government benefits to enforcement of the law, monitoring of risks, and provision of services. Despite technological improvements, AI systems are fallible and may err. How do consumers respond when learning of AI’s failures? In thirteen preregistered studies (N = 3,724) across policy areas, we show that algorithmic failures are generalized more broadly than human failures. We term this effect algorithmic transference, as it is an inferential process that generalizes (i.e., transfers) information about one member of a group to another member of that same group. Rather than reflecting generalized algorithm aversion, algorithmic transference is rooted in social categorization: it stems from how people perceive a group of AI systems versus a group of humans. Because AI systems are perceived as more homogeneous than people, failure information about one AI algorithm is transferred to another algorithm at a higher rate than failure information about a person is transferred to another person. Assessing AI’s impact on consumers and societies, we show how the premature or mismanaged deployment of faulty AI technologies may undermine the very institutions that AI systems are meant to modernize.

Relevance insensitivity: A framework of psychological biases in consumer behavior and beyond

“…Consequently, raising awareness of algorithmic racial bias can deter Black consumers, but not White consumers, from using “good” (i.e., fair and beneficial) algorithms. Relatedly, Longoni et al, (forthcoming) found that people are more likely to generalize algorithmic failures than human failures.…”