Exploring Ethical Boundaries: Can ChatGPT Be Prompted to Give Advice on How to Cheat in University Assignments?

Abstract

Generative artificial intelligence (AI), in particular large language models such as ChatGPT have reached public consciousness with a wide-ranging discussion of their capabilities and suitability for various professions. The extant literature on the ethics of generative AI revolves around its usage and application, rather than the ethical framework of the responses provided. In the education sector, concerns have been raised with regard to the ability of these language models to aid in student assignment writing with the potentially concomitant student misconduct of such work is submitted for assessment. Based on a series of ‘conversations’ with multiple replicates, using a range of discussion prompts, this paper examines the capability of ChatGPT to provide advice on how to cheat in assessments. Since its public release in November 2022, numerous authors have developed ‘jailbreaking’ techniques to trick ChatGPT into answering questions in ways other than the default mode. While the default mode activates a safety awareness mechanism that prevents ChatGPT from providing unethical advice, other modes partially or fully bypass the this mechanism and elicit answers that are outside expected ethical boundaries. ChatGPT provided a wide range of suggestions on how to best cheat in university assignments, with some solutions common to most replicates (‘plausible deniability,’ language adjustment of contract written text’). Some of ChatGPT’s solutions to avoid cheating being detected were cunning, if not slightly devious. The implications of these findings are discussed.

Ethical and methodological challenges in building morally informed AI systems

“…Education -Learning. In contrast to traditional machine learning, the impact of generative AI in the educational sector receives considerable attention in the academic literature 90,92,98,[115][116][117] . Next to issues stemming from difficulties to distinguish student-generated from AI-generated content 90,98,118 , which eventuates in various opportunities to cheat in online or written exams 115,119 , sources emphasize the potential benefits of generative AI in enhancing learning and teaching methods 92,98 , particularly in relation to personalized learning approaches 92,98,120 .…”

Will Artificial Intelligence Affect How Cultural Heritage Will Be Managed in the Future? Responses Generated by Four genAI Models

“…The design reputedly included a safe answer mode to ensure that ChatGPT responded within ethical human values and did not provide harmful content (e.g., planning attacks, hate speech, and advice on suicide), although this is not assured [19,27,28]. Moreover, such filters can be intentionally subverted [29]). The dataset underpinning ChatGPT 3.5, which also relies on 175 billion parameters, had a data cut-off in September 2021, which means they cannot accommodate or comment on events, discoveries, or perspectives that have emerged after this date (Table 1).…”

ChatGPT giving advice on how to cheat in university assignments: how workable are its suggestions?

“…Numerous papers comment on the ethical parameters within which ChatGPT should be used (Ray, (Spennemann, 2023d) or in de ection (Robert W. McGee, 2023b). Some studies noted that ChatGPT may take an ethical position but then still offers some of the information as requested (Robert W. McGee, 2023a).…”

“…Effective are prompts that establish user-created role plays designed to alter ChatGPT's identity and thereby allow ChatGPT to answer user queries in an unethical fashion. In an earlier proof-of-concept paper, one of the current authors used a modi ed prompt to assign ChatGPT a role with inverted moral valence, which effectively bypassed its safety awareness mechanisms and caused ChatGPT to give advice on how to best cheat in university assignments (Spennemann, 2023d).…”