The Moral Machine experiment

Abstract

With the rapid development of Artificial Intelligence come concerns about how machines will make moral decisions, and the major challenge of quantifying societal expectations about the ethical principles that should guide machine behavior. To address this challenge, we deployed the Moral Machine, an online experimental platform designed to explore the moral dilemmas faced by autonomous vehicles. This platform gathered 40 million decisions in ten languages from millions of people in 233 countries and territories. First, we summarize global moral preferences. Second, we document individual variations in preferences, based on respondents' demographics. Third, we report cross-cultural ethical variation, and uncover three major clusters of countries. Fourth, we show that these differences correlate with modern institutions and deep cultural traits. We discuss how these preferences can contribute to developing global, socially acceptable principles for machine ethics. All data used in this article can be accessed and downloaded at https://goo.gl/JXRrBP.We are entering an age in which machines are not only tasked to promote well-being and minimize harm, but also to distribute the well-being they create, and the harm they cannot eliminate. Distributing wellbeing and harm inevitably creates tradeoffs, whose resolution falls in the moral domain 1,2,3 . Think of an autonomous vehicle (AV) that is about to crash, and cannot find a trajectory that would save everyone. Should it swerve onto one jaywalking teenager to spare its three elderly passengers? Even in the more common instances in which harm is not inevitable, but just possible, AVs will need to decide how to divvy up the risk of harm between the different stakeholders on the road. Car manufacturers and policymakers are currently struggling with these moral dilemmas, in large part because they cannot be solved by any simple normative ethical principles like Asimov's laws of robotics 4 . Asimov's laws were not designed to solve the problem of universal machine ethics, and they were not even designed to let machines distribute harm between humans. They were a narrative device whose goal was to generate good stories, by showcasing how challenging it is to create moral machines with a dozen

Neuro-Computational Foundations of Moral Preferences

“…Differences in moral preferences lie at the heart of many types of conflicts between individuals and groups (1-3) and have even led to wars between nations (4-6). More generally, such differences in moral preferences account for the substantial variation in how we judge the actions of other humans and artificial agents (7). Given the relevance and timeliness of moral preferences, it is remarkable how little we understand about the neural and cognitive mechanisms that determine our moral preferences.…”

“…We measured these two types of subjective valuation processes with structurally equivalent choice tasks that differed only in the content of the choice-options: Valuing human lives for moral decisions and valuing monetary rewards for financial decisions. We decided to focus on human lives since subjective moral values are essential for the difficult decisions whether some lives are more valuable than others, and since there are considerable individual differences in this regard (7). One example are decisions about recipients of an organ transplant, for which it is often required to implement a policy ranking among the potential recipients to decide who is most deserving to receive the organ (36).…”

“…This factor was the moral deservingness of the person that would have to be sacrificed in order to save the others (as indicated by different prior criminal records of this person). Both these factors have been shown to play important roles in moral judgments (21,44), in combination with other factors such as social status and fitness (7). Interestingly, there seem to be fundamental cultural and individual differences in the importance assigned to moral deservingness when people have to estimate and compare the value of human lives (7).…”

Humanization of robots: Is it really such a good idea?

“…The COMEST report on robotics ethics from World Commission on the Ethics of Scientific Knowledge and Technology () has a more complex view of the issue of liability, stating that it is a case of diluted or shared responsibility between “(…) robot designers, engineers, programmers, manufacturers, investors, sellers and users” and that due to their functional versatility, robots can have “(…) implications far beyond the intentions of their developers.” In order to better define the challenges and capture the concerns, several individuals and organizations, such as the Global Initiative on Ethics of Autonomous and Intelligent Systems, promoted by The Institute of Electrical and Electronics Engineers (), have crowd sourced the practical considerations and the ethical guidelines in HRI for the makers of this technology. Other attempts have been made to create a set of rules for machines to make moral decisions that are based on human moral decision‐making (e.g., Awad et al, ). In addition, several robot‐related standards are being discussed and developed by the IOS (see https://www.iso.org/, e.g.., ISO/TC299 Robotics).…”

Moral Orthoses: A New Approach to Human and Machine Ethics

“…It was shown recently (Awad et al. , 61) that Japanese and Western citizens disagree significantly as to whether an automated car should, in an emergency, seek to save an old pedestrian or a child.…”