MIT’s moral machine project is a psychological roadblock to self-driving cars

Abstract

In the moral machine project, participants are asked to form judgments about the well-known trolley example. The project is intended to serve as a starting point for public discussion that would eventually lead to a solution to the social dilemma of autonomous vehicles. The dilemma is that autonomous vehicles should be programed to maximize the number of lives saved in trolley-style dilemmas. But consumers will only purchase autonomous vehicles that are programed to favor passenger safety in such dilemmas. We argue that the project is seriously misguided. There are relevant variants of trolley to which the project's participants are not exposed. These variants make clear that the morally correct way to program autonomous vehicles is not at odds with what consumers will purchase. The project is hugely popular and dominates public discussion of this issue. We show that, ironically, the project itself is largely responsible for the dilemma. Keywords Moral machine project • Trolley problem • Autonomous vehicles In the moral machine project 1 , participants are asked to form judgments about variations of this well-known example: Trolley: There is a runaway trolley. If you do nothing, the trolley will hit and kill five people. If you pull a lever, the trolley will be diverted onto another track and kill one person.

Coordinated Control Design for Ethical Maneuvering of Autonomous Vehicles

“…The analysis contributed to the empirical determination of ethical preferences, their priorities, and their dependence on individual, cultural, and demographic characteristics [10]. However, the results of the study can also pose psychological obstacles to the widespread adoption of automated vehicles, as the expectation of vehicle ownership and the need to maximize the number of lives saved in accidents may conflict with each other [11]. Similarly, the public is willing to express more blame for automated vehicles as for human drivers in a mixed traffic crash situation [12].…”

Trolleys, crashes, and perception—a survey on how current autonomous vehicles debates invoke problematic expectations

“…People accept and prefer other people to buy utilitarian AVs but in personal use they would like to use the ones that save their own lives [ 30 ]. This distinction in the expected behavior of AVs could even lead to a decrease in the overall acceptance of such vehicles [ 31 ].…”

The moral machine experiment on large language models

“…The findings from the MM experiment suggest a discernible trend favouring the preservation of human lives over animals, emphasizing the protection of a greater number of lives and prioritizing the safety of the young. Although the MM experiment has significant limitations in its applicability [ 13 – 15 ], and we must be careful when interpreting the results of the MM experiment [ 16 , 17 ], these preferences are seen as foundational to machine ethics and essential considerations for policymakers [ 18 ]. The insights gained from this study emphasize the importance of aligning AI ethical guidelines with human moral values.…”