Exploring the roles of trust and social group preference on the legitimacy of algorithmic decision-making vs. human decision-making for allocating COVID-19 vaccinations

Abstract

In combating the ongoing global health threat of the COVID-19 pandemic, decision-makers have to take actions based on a multitude of relevant health data with severe potential consequences for the affected patients. Because of their presumed advantages in handling and analyzing vast amounts of data, computer systems of algorithmic decision-making (ADM) are implemented and substitute humans in decision-making processes. In this study, we focus on a specific application of ADM in contrast to human decision-making (HDM), namely the allocation of COVID-19 vaccines to the public. In particular, we elaborate on the role of trust and social group preference on the legitimacy of vaccine allocation. We conducted a survey with a 2 × 2 randomized factorial design among n = 1602 German respondents, in which we utilized distinct decision-making agents (HDM vs. ADM) and prioritization of a specific social group (teachers vs. prisoners) as design factors. Our findings show that general trust in ADM systems and preference for vaccination of a specific social group influence the legitimacy of vaccine allocation. However, contrary to our expectations, trust in the agent making the decision did not moderate the link between social group preference and legitimacy. Moreover, the effect was also not moderated by the type of decision-maker (human vs. algorithm). We conclude that trustworthy ADM systems must not necessarily lead to the legitimacy of ADM systems.

Algorithmic discrimination: examining its types and regulatory measures with emphasis on US legal practices

“…This includes recalibrating the algorithm, re-selecting features, re training the model, or adopting other methods to ensure that the algorithm does not produce discriminatory results. The purpose of these measures is to ensure that the algorithm does not adversely affect certain groups in practical application, even if this is not the explicit intention of the users of the algorithm ( Lünich and Kieslich, 2024 ).…”

Ever heard of ethical AI? Investigating the salience of ethical AI issues among the German population

“…Additionally, journalists and activists also raised concerns about the implementation of unethical AI systems, for example in Sweden in the use case of a social benefit application and in Germany in the use case of face recognition in public places (Algorithm Watch, 2020). Additional studies suggest that the perceptions of unfair treatment by ADM systems can lead to the rejection of such technology and of those who apply it, respectively (Marcinkowski, Kieslich, Starke, & Lünich, 2020); furthermore, Lünich and Kieslich (2022) noted that a lack of trust in ADM systems leads to them being perceived as illegitimate. However, Algorithm Watch estimates the awareness of ethical problems with AI as not being overly high: "Protests in the UK and elsewhere, together with high-profile scandals based on ADM systems, have certainly raised awareness of both the risks and opportunities of automating society.…”

“…Also, the cases were protest was articulated by the population or by civil society actors were tied to a specific example, e.g. education admission systems (Kelly, 2021) or vaccine allocation (Lünich & Kieslich, 2022). As the literature does not suggest, which use cases are currently on the top of the head of citizens or especially suitable for a public debate about ethical issues of AI, we formulated H2 rather broad: H2: People who are concerned with ethical AI commonly connect AI to specific applications.…”

QR codes and automated decision-making in the COVID-19 pandemic

“…Digital health played a critical role in the COVID-19 response from the outset. An overarching challenge posed by a pandemic for authorities was how to quickly and accurately obtain health status information on individuals, and make risk, safety, and health care decisions (Lünich and Kieslich, 2022). Given the lethal nature of COVID-19, as well as the enormous health and impairment burden it rapidly entailed, the need to make such decisions as quickly as possible, if not in-near-real-time, and at scale (population-wide as well as for specific groups and locations) was evident.…”