People Prefer Moral Discretion to Algorithms: Algorithm Aversion Beyond Intransparency

Abstract

We explore aversion to the use of algorithms in moral decision-making. So far, this aversion has been explained mainly by the fear of opaque decisions that are potentially biased. Using incentivized experiments, we study which role the desire for human discretion in moral decision-making plays. This seems justified in light of evidence suggesting that people might not doubt the quality of algorithmic decisions, but still reject them. In our first study, we found that people prefer humans with decision-making discretion to algorithms that rigidly apply exogenously given human-created fairness principles to specific cases. In the second study, we found that people do not prefer humans to algorithms because they appreciate flesh-and-blood decision-makers per se, but because they appreciate humans’ freedom to transcend fairness principles at will. Our results contribute to a deeper understanding of algorithm aversion. They indicate that emphasizing the transparency of algorithms that clearly follow fairness principles might not be the only element for fostering societal algorithm acceptance and suggest reconsidering certain features of the decision-making process.

Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

“…They have neither yet been validated for specific applications based on real data nor confirmed by including end users to unearth their true pertinency towards said tradeoff between performance vs. explainability. An empirical quantification of end user explainability is necessary to provide first-hand knowledge to the engineers of intelligent systems for the development of intelligent systems (Jauernig et al, 2022;. Despite this apparent deficiency, they are commonly referenced as a motivation for a user-or organization-centered XAI research or intelligent system deployment (e. g., Asatiani et al, 2021;Guo et al, 2019;Rudin, 2019).…”

“…The dependent variable performance measures the objective performance of the algorithm. The perceived goodness of explanation is more subjective and we base the choice of this second dependent variable on the proposed tradeoff that requires a quantification of explanation as it is perceived by users and knowingly can influence the user's mindset towards algorithms (Berger et al, 2021;Jauernig et al, 2022). The moderating group variable data complexity is expressed through different cases using different datasets reflecting low complexity and high complexity.…”

AI Trust: Can Explainable AI Enhance Warranted Trust?

“…In contexts of human-AI collaboration, the perception of an AI system has been found to be influenced by several variables, including communication direction and the nature of the model underpinning the AI system [28]. Beyond the usual human aversion to algorithms [29], research indicates that individuals prefer human decision-making discretion over algorithms that rigidly apply human-derived fairness principles to specific cases [30]. This preference stems from humans' capacity for independent judgment, allowing them to transcend fairness principles as necessary.…”

AI, Opacity, and Personal Autonomy

“…8 London (2019) emphasizes that common medical interventions often involve mechanisms that are not 6 Although there are definitely other factors in play as well. See for example Krügel et al (2022) and Jauernig et al (2022).…”

“…More on this later.6 Although there are definitely other factors in play as well. See for exampleKrügel et al (2022) andJauernig et al (2022).7 Though seeWeller (2019, Section 3.1) for examples suggesting that transparency can lead us to overascribe reliability.…”