People are averse to machines making moral decisions

Bigman, Yochanan E.; Gray, Kurt

doi:10.1016/j.cognition.2018.08.003

Cited by 339 publications

(318 citation statements)

References 70 publications

Supporting

Mentioning

286

Contrasting

Unclassified

Order By: Relevance

“…Moreover, the human nurse was generally trusted more than the robot nurse, and this effect was slightly stronger when the patient's autonomy was violated. Together, these results strongly suggest that, in our context, decisions involving AIs are morally condemnable (compared with similar decisions made by humans) only when in violation of personal autonomy; this raises some tensions between our results and those of Bigman and Gray (2018), which suggest that the aversion to robot decisions is driven by mind perception.…”

Section: Discussion Of Studysupporting

confidence: 49%

“…Moral psychology of robotics and artificial intelligence (AI) is a new growing field of research on moral cognition (Awad et al, 2018, Malle et al, 2015, Bonnefon et al, 2016. In the past few years, several articles have examined the moral relations between humans and new emerging AI technologies from different angles; such as the moral psychological aspects of robot prostitution (Koverola et al, in press), self-driving vehicles (Awad et al, 2018, Malle et al, 2015, Bonnefon et al, 2016, or the moral interaction in medical decision-making between robots and humans (Bigman & Gray, 2018). Recent studies have also drawn from transhumanist themes and examined the role of sacredness and sexual disgust in condemning mind upload technologies , and human attitudes towards brain enhancement technologies (Medaglia, Yaden, Helion & Haslam, 2019).…”

Section: Moral Psychology Of Nursing Robots -Humans Dislike Violationmentioning

confidence: 99%

“…In a recent study, Bigman and Gray (2018) found that people were generally averse to robots making moral decisions, including medical ones. We expected to conceptually replicate this finding, but also examined a variety of contextual factors involved in medical decision-making situations.…”

Section: Moral Psychology Of Nursing Robots -Humans Dislike Violationmentioning

confidence: 99%

See 2 more Smart Citations

Moral Psychology of Nursing Robots: Humans Dislike Violations of Patient Autonomy but Like Robots Disobeying Orders

Laakasuo¹,

Kunnari²,

Palomäki³

et al. 2019

Preprint

View full text Add to dashboard Cite

Artificial intelligences (AIs) are widely used in tasks ranging from transportation and healthcare to military. Many tasks carried out by autonomous AIs have consequences for human well-being, but it is still unclear how people would prefer them to act in ethically difficult situations. In six studies with data from two cultures (five quantitative experiments, n = 1569, and a qualitative anthropological field study, n = 30), we presented people with hypothetical situations where a human or an advanced robot nurse is ordered to forcefully medicate an unwilling patient. We measured moral acceptance, perceived trust, and allocation of responsibility relating to the nurse's decision of either following orders to forcefully medicate the patient, or disregarding orders to protect the patient's autonomy. Our participants were aversive to robot nurses who forcefully medicated the patient, and preferred robot nurses who respected patient autonomy by disobeying orders. Under certain conditions, the decision to respect patient autonomy was more acceptable for robot nurses than for human nurses. Thus, our results suggest that people prefer robots that are capable of disobeying orders in favor of abstract moral principles such as valuing personal autonomy. These findings were relatively robust against manipulating the nurse's perceived reputation and character, and whether or not the patient lived or died afterwards. We also found that moral judgment is distinct from evaluations of trust and responsibility. In general, our participants did not trust robot nurses or hold them responsible for their actions; on the other hand human nurses who forcefully medicated a patient were morally condemned but also trusted. It seems that Moral Psychology of Robotics is a new and increasingly relevant sub-field of moral psychology that requires extensive attention.

show abstract

Section: Discussion Of Studysupporting

confidence: 49%

Section: Moral Psychology Of Nursing Robots -Humans Dislike Violationmentioning

confidence: 99%

See 1 more Smart Citation

Moral Psychology of Nursing Robots: Humans Dislike Violations of Patient Autonomy but Like Robots Disobeying Orders

Laakasuo¹,

Kunnari²,

Palomäki³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Similarly, Złotowski, Yogeeswaran, and Bartneck () found that when robots were depicted in a video as being autonomous they elicited more negative attitudes and perceptions of threat than robots that were non‐autonomous. Interestingly, when asked about whether robots should make moral decisions (e.g., deciding on the punishment for a convicted criminal), people are uncomfortable with robots making such decisions because they are perceived to lack complete minds (Bigman & Gray, ).…”

Section: Robots As a Threatmentioning

confidence: 99%

“Danger, Will Robinson!” The challenges of social robots for intergroup relations

Vanman

Kappas

2019

Social & Personality Psych

View full text Add to dashboard Cite

Society's increasing reliance on robots in everyday life provides exciting opportunities for social psychologists to work with engineers in the nascent field of social robotics. In contrast to industrial robots that, for example, may be used on an assembly line, social robots are designed specifically to interact with humans and/or other robots. People tend to perceive social robots as autonomous and capable of having a mind. As such, they are also more likely to be subject to social categorization by humans. As social robots become more human like, people may also feel greater empathy for them and treat robots more like (human) ingroup members. On the other hand, as they become more human like, robots also challenge our human distinctiveness, threaten our identity, and elicit suspicion about their ability to deceive us with their human‐like qualities. We review relevant research to explore this apparent paradox, particularly from an intergroup relations perspective. We discuss these findings and propose three research questions that we believe social psychologists are ideally suited to address.

show abstract

“…Furthermore, since self-awareness may lead to the development of a theory of mind (see the last section), a self-aware and "mentalizing" robot could better cooperate with humans and other AI agents. Bigman and Gray (2018) suggest that increasing elements of robot self-awareness as the theory of mind, situation awareness, intention, free will, could serve as a foundation for increasing human trust in robot autonomy because humans tend to judge the role of these and other perceived mental faculties as necessary in autonomy.…”

Section: Why Would Self-awareness Benefit Robots?mentioning

confidence: 99%

Developing Self-Awareness in Robots via Inner Speech

et al. 2020

View full text Add to dashboard Cite

The experience of inner speech is a common one. Such a dialogue accompanies the introspection of mental life and fulfills essential roles in human behavior, such as self-restructuring, self-regulation, and re-focusing on attentional resources. Although the underpinning of inner speech is mostly investigated in psychological and philosophical fields, the research in robotics generally does not address such a form of self-aware behavior. Existing models of inner speech inspire computational tools to provide a robot with this form of self-awareness. Here, the widespread psychological models of inner speech are reviewed, and a cognitive architecture for a robot implementing such a capability is outlined in a simplified setup.

show abstract

People are averse to machines making moral decisions

Cited by 339 publications

References 70 publications

Moral Psychology of Nursing Robots: Humans Dislike Violations of Patient Autonomy but Like Robots Disobeying Orders

Moral Psychology of Nursing Robots: Humans Dislike Violations of Patient Autonomy but Like Robots Disobeying Orders

“Danger, Will Robinson!” The challenges of social robots for intergroup relations

Developing Self-Awareness in Robots via Inner Speech

Contact Info

Product

Resources

About