Deceptive AI Ecosystems: The Case of ChatGPT

Abstract

ChatGPT, an AI chatbot, has gained popularity for its capability in generating human-like responses. However, this feature carries several risks, most notably due to its deceptive behaviour such as offering users misleading or fabricated information that could further cause ethical issues. To better understand the impact of ChatGPT on our social, cultural, economic, and political interactions, it is crucial to investigate how ChatGPT operates in the real world where various societal pressures influence its development and deployment. This paper emphasizes the need to study Chat-GPT "in the wild", as part of the ecosystem it is embedded in, with a strong focus on user involvement. We examine the ethical challenges stemming from ChatGPT's deceptive human-like interactions and propose a roadmap for developing more transparent and trustworthy chatbots. Central to our approach is the importance of proactive risk assessment and user participation in shaping the future of chatbot technology. CCS CONCEPTS• Human-centered computing → Natural language interfaces; HCI theory, concepts and models.

Ethical and methodological challenges in building morally informed AI systems

“…Additionally, papers explore the potential for generative models to aid in criminal activities 82 , incidents of self-harm 72 , identity theft 35 , or impersonation 83 . Furthermore, the literature investigates risks posed by LLMs when generating advice in high-stakes domains such as health 84 , safety-related issues 85 , as well as legal or financial matters 86 .…”

“…Papers not only critically analyze various types of reasoning errors in LLMs 87 but also examine risks associated with specific types of misinformation, such as medical hallucinations 97 . Given the propensity of LLMs to produce flawed outputs accompanied by overconfident rationales 93 and fabricated references 86 , many sources stress the necessity of manually validating and fact-checking the outputs of these models 92,98,99 .…”

“…Apart from that, another minor topic area concerns responsible approaches of talking about generative AI systems 108 . This includes avoiding overstating the capabilities of generative AI 60 , reducing the hype surrounding it 86 , or evading anthropomorphized language to describe model capabilities 35 .…”

Caution with AI deception: AI could serve as the scapegoat of Human!

“…Trust also plays a crucial role in human-AI interactions. Individuals' trust in AI may be compromised if they feel deceived (Evans et al, 2021;Zhan et al, 2023). In a chess puzzle task where participants collaborated with an AI teammate, participants were less likely to accept the AI teammate's decisions if they were deceived into thinking they were working with another human rather than an AI (Zhang et al, 2023).…”

“…When AI systems generate false or misleading information that appears authentic and reliable, humans may unknowingly accept and propagate it. The possibility of AI producing misleading and deceptive information on a large scale to users is a serious concern, as it could have adverse impacts on those unable to distinguish between 'fact' and 'fiction', leading to detrimental consequences (Zhan et al, 2023). While deception in human contexts has received extensive study, empirical research about deception of artificial intelligence remains relatively underexplored.…”