Moral Psychology of Nursing Robots: Humans Dislike Violations of Patient Autonomy but Like Robots Disobeying Orders

Abstract

Artificial intelligences (AIs) are widely used in tasks ranging from transportation and healthcare to military. Many tasks carried out by autonomous AIs have consequences for human well-being, but it is still unclear how people would prefer them to act in ethically difficult situations. In six studies with data from two cultures (five quantitative experiments, n = 1569, and a qualitative anthropological field study, n = 30), we presented people with hypothetical situations where a human or an advanced robot nurse is ordered to forcefully medicate an unwilling patient. We measured moral acceptance, perceived trust, and allocation of responsibility relating to the nurse's decision of either following orders to forcefully medicate the patient, or disregarding orders to protect the patient's autonomy. Our participants were aversive to robot nurses who forcefully medicated the patient, and preferred robot nurses who respected patient autonomy by disobeying orders. Under certain conditions, the decision to respect patient autonomy was more acceptable for robot nurses than for human nurses. Thus, our results suggest that people prefer robots that are capable of disobeying orders in favor of abstract moral principles such as valuing personal autonomy. These findings were relatively robust against manipulating the nurse's perceived reputation and character, and whether or not the patient lived or died afterwards. We also found that moral judgment is distinct from evaluations of trust and responsibility. In general, our participants did not trust robot nurses or hold them responsible for their actions; on the other hand human nurses who forcefully medicated a patient were morally condemned but also trusted. It seems that Moral Psychology of Robotics is a new and increasingly relevant sub-field of moral psychology that requires extensive attention.

Expectations towards the Morality of Robots: An Overview of Empirical Studies

“…Table 3 presents a summary of the selected studies, the one presented in Section 2 (see also Laakasuo et al 2019;Malle et al 2015). All three studies subject to this analysis employed stories of robots acting in hypothetical scenarios.…”

“…The task of the respondents was to assess the rightness or moral acceptance of the robots and humans. Additional measures included: the deserved blame (Malle et al 2015); the moral responsibility, and the trust of the agent presented in the story (Laakasuo et al 2019). Malle et al (2015) and the 6 th experiment of Laakasuo et al (2019) also incorporated a request for justification of the answers given.…”

“…Malle et al (2015) presented an "advanced state-of-the-art" repair robot working for the railways. The robot introduced by Laakasuo et al (2019) was an advanced nursing robot. Such a description of robots was used to assign one more important feature to them.…”

“…A short story where a human or a robot nurse is ordered to give a patient a medication against the patient's will was introduced in Laakasuo et al (2019). The moral acceptance of forcefully medicating the patient was lower if done by the robot nurse compared to a human nurse.…”

“…The following section evaluates Asimov's Three Laws of Robotics and examines the relationship between personal moral beliefs and an ethical evaluation of other people and robots' attitudes. These results are then compared to two selected studies that use an analogical methodological approach, namely "Moral psychology of nursing robots -humans dislike violations of patient autonomy but like robots disobeying orders" (Laakasuo, Kunnari, Palomäki, Rauhala, Koverola, Lehtonen, Halonen, Repo, Visala, & Drosinou 2019) and "Sacrifice one for the good of many? : People apply different moral norms to human and robot agents" (Malle, Scheutz, Arnold, Voiklis, & Cusimano 2015).…”