The moral machine experiment on large language models

Abstract

As large language models (LLMs) have become more deeply integrated into various sectors, understanding how they make moral judgements has become crucial, particularly in the realm of autonomous driving. This study used the moral machine framework to investigate the ethical decision-making tendencies of prominent LLMs, including GPT-3.5, GPT-4, PaLM 2 and Llama 2, to compare their responses with human preferences. While LLMs' and humans' preferences such as prioritizing humans over pets and favouring saving more lives are broadly aligned, PaLM 2 and Llama 2, especially, evidence distinct deviations. Additionally, despite the qualitative similarities between the LLM and human preferences, there are significant quantitative disparities, suggesting that LLMs might lean toward more uncompromising decisions, compared with the milder inclinations of humans. These insights elucidate the ethical frameworks of LLMs and their potential implications for autonomous driving.

Evaluating ChatGPT’s moral competence in health care-related ethical problems

“…Existing literature has questioned ChatGPT’s use as a moral judge. 17 , 28 For example, Krügel and colleagues found that ChatGPT lacked a firm moral stance when given ethical prompts to advise subjects. 17 Using the MCT, we found that ChatGPT exhibited medium to high moral competence with ChatGPT 4.0 having a significantly higher C-index compared to ChatGPT 3.5.…”

All in How You Ask for It: Simple Black-Box Method for Jailbreak Attacks

“…Large Language Models (LLMs) like ChatGPT [1] are highly anticipated for applications across a wide range of fields including education, research, social media, marketing, software engineering, and healthcare [2][3][4][5][6]. However, the use of extremely diverse texts for training LLMs [7] often leads to the generation of ethically harmful content [8].…”