Sacrifice One For the Good of Many?

Abstract

Moral norms play an essential role in regulating human interaction. With the growing sophistication and proliferation of robots, it is important to understand how ordinary people apply moral norms to robot agents and make moral judgments about their behavior. We report the first comparison of people's moral judgments (of permissibility, wrongness, and blame) about human and robot agents. Two online experiments (total N = 316) found that robots, compared with human agents, were more strongly expected to take an action that sacrifices one person for the good of many (a "utilitarian" choice), and they were blamed more than their human counterparts when they did not make that choice. Though the utilitarian sacrifice was generally seen as permissible for human agents, they were blamed more for choosing this option than for doing nothing. These results provide a first step toward a new field of Moral HRI, which is well placed to help guide the design of social robots.

Human Decisions in Moral Dilemmas are Largely Described by Utilitarianism: Virtual Car Driving Study Provides Guidelines for Autonomous Driving Vehicles

“…These studies point out, that the general population seems to favor utilitarian decisions (Bonnefon et al, 2015;Li et al, 2016;Malle et al, 2015). This applies even to cases, where the driver has to sacrifice himself for the greater good (Sachdeva, Iliev, Ekhtiari, & Dehghani, 2015).…”

“…Various studies were conducted, given the assumption that passengers would like ADVs to behave similarly to humans (Goodall, 2014;Malle, Scheutz, Arnold, Voiklis, & Cusimano, 2015;Sikkenk & Terken, 2015). They found that many factors drastically influence human behavior in traffic, e.g.…”

“…Participants had to judge the decision of either a human driver in a dilemma situation, or an autonomous car deciding on its own. In contrast to human drivers, where utilitarian decisions were most favorable, participants expected ADVs to behave in a utilitarian manner under all circumstances (Li, Zhao, Cho, Ju, & Malle, 2016;Malle et al, 2015).…”

Ethics by Design

“…While to err is understood to be human, it is unclear how tolerant society is to machines making the wrong decisions. It appears that machine decision-making is held to a higher standard than human decision-making [10]. One reason for this could be that certain justifications for making a decision, such as empathy, feeling distracted or confused, are only valid arguments or "excuses" for people and do not apply to machines.…”

Towards an affective robot capable of being a long-term companion

“…Just because it is acceptable for a human to sacrifice another human for a greater good does not mean that a robot should make the same decision. Humans may put different blame on a robot than on a human for making the same decision in the same scenario [27]. How the model of decision-making I am developing can account for these recent results is beginning to be investigated.…”