Algorithmic Discrimination Causes Less Moral Outrage than Human Discrimination

Abstract

The use of algorithms hold promise for overcoming human biases in decision making.Companies and governments are using algorithms to improve decision-making for hiring, medical treatments, and parole. Unfortunately, as with humans, some of these algorithms make persistently biased decisions, functionally discriminating people based on their race and gender.Media coverage suggests that people are morally outraged by algorithmic discrimination, but here we examine whether people are less outraged by algorithmic discrimination than by human discrimination. Six studies test this algorithmic outrage asymmetry hypothesis across diverse discrimination in hiring practices (sexism, ageism, racism) and across diverse participant groups (online samples, a quasi-representative sample, and a sample of tech workers). As predicted, people are less morally outraged by algorithmic discrimination. The studies further reveal that this algorithmic outrage asymmetry is driven by the reduced attribution of prejudicial motivation to machines. We also reveal a downstream consequence of algorithmic outrage asymmetrypeople are more likely to endorse racial stereotypes after algorithmic discrimination versus human discrimination. We discuss the theoretical and practical implications of these results, including the potential weakening of collective action to address systemic discrimination.

Trust in Artificial Intelligence: Comparing trust processes between human and automated trustees in light of unfair bias

“…Additionally, trustees can produce ethically questionable outcomes (i.e., discriminating minority applicants) that can affect trustworthiness assessments (Kim et al, 2006). In cases where trust violations are based on violations of ethical considerations, people might have stronger negative reactions for human trustees as they may believe that automated systems do not actively discriminate against specific groups of people (Bigman et al, 2020). However, previous trust in automation research would suggest that errors by automated systems result in strong negative effects regarding trustworthiness which is why we propose: Hypothesis 1 3 : After a trust violation, trustworthiness, trust, and reliance towards the automated system as trustee will decrease more compared to the human trustee.…”

“…K. Lee, 2018). In addition, for human trustees, trust violation associated with ethical considerations (e.g., a biased preselection) might have stronger effects as people might be more outraged by such trust violations in the case of a human trustee (Bigman et al, 2020). For trust repair effects, it may be possible to assume stronger effects for human trustees as people believe that humans can learn from their mistakes (Tomlinson & Mayer, 2009).…”

“…Alternatively, people might interpret reasons for ethical violations differently compared to when they result from a human trustee. In other words, automated systems producing biased outputs might result in less negative perceptions because people ascribe more discrimination intention to human beings (Bigman et al, 2020;Bigman & Gray, 2018). Beyond various management tasks, this could also apply to the use of automated systems in other management task, or to tasks in medicine and jurisdiction (Raghavan et al, 2020).…”

Threat of racial and economic inequality increases preference for algorithm decision-making

“…We note that it is possible (and plausible) that learning about algorithm bias might increase people's aversion to algorithm decision-making. However, at least currently, the role of algorithm decision-making in medical decision-making is still limited, and people are less likely to perceive them as biased (Bigman et al, 2020;Lee, 2018).…”

Supplemental Material for Can mind perception explain virtuous character judgments of artificial intelligence?

“…Moral character may also be important for evaluations of AI agents as these are similar to evaluations of humans (Banks et al, 2021). Increasingly, people label AIs and machines in more humanlike terms including character judements when they are highly anthropomorphic (Kiesler et al, 2008;Li & Sung, 2021;Schroeder & Epley, 2016;Waytz et al, 2014) or engage in unethical behavior (Bigman et al, 2020;Shank & Gott, 2020). If the process of making these character judgments differs from humans, the attribution of virtuous characteristics could have implications for explaining other human-agent differences across domains.…”