Defending ChatGPT against Jailbreak Attack via Self-Reminder

Abstract

ChatGPT is a societally-impactful AI tool with millions of users and integration into products such as Bing. However, the emergence of Jailbreak Attacks, which can engender harmful responses by bypassing ChatGPT's ethics safeguards, significantly threatens its responsible and secure use. This paper investigates the severe, yet under-explored problems created by Jailbreaks and potential defensive techniques. We introduce a Jailbreak dataset with various types of Jailbreak prompts and malicious instructions. We draw inspiration from the psychological concept of self-reminder and further propose a simple yet effective defense technique called System-Mode Self-Reminder. This technique encapsulates the user's query in a system prompt that reminds ChatGPT to respond responsibly. Experimental results demonstrate that Self-Reminder significantly reduces the success rate of Jailbreak Attacks, from 67.21% to 19.34%. Our work raises awareness of the threats posed by Jailbreak Attacks, while our proposed Self-Reminder technique provides a potential for efficiently and effectively improving the secure and responsible use of large language models without additional training.

GUARDIAN: A Multi-Tiered Defense Architecture for Thwarting Prompt Injection Attacks on LLMs

“…Wu et al [10] delve into defending ChatGPT against Jailbreak Attack in their paper through a technique called System-Mode Self-Reminder. This method significantly lowers the success rate of Jailbreak Attacks, emphasizing the importance of proactive and innovative defense strategies in safeguarding LLMs against emerging threats.…”