ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models

Abstract

Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI's ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny?We put ChatGPT's sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT's capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments.Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead Chat-GPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward "funny" machines.

“Funny How?” A Serious Look at Humor in Conversational Agents

“…This may represent a new opportunity for humor in CAs. Yet, despite this technical opportunity, large language models seem to mainly use pre-scripted humor to this point, as Jentzsch and Kersting [26] found. Their study revealed that 90% of the 1008 jokes they generated with ChatGPT were the same 25 jokes.…”

“…To avoid these risks, companies must exercise caution and consider the impact their content can have. This could also be a reason why LLMs are constrained to certain pre-scripted humorous interactions Jentzsch and Kersting [26]. The risk or fear of being offensive can be an important factor in why CA companies prefer to play it safe with harmless and corny jokes.…”

Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association?

“…humor, is important during communication. Ope-nAI (2023) has reported GPT-4's ability to understand jokes, while subsequent works show limited capacity for LLMs to generate or explain humor (Jentzsch and Kersting, 2023;Hessel et al, 2023). As creativity is essential to the development of art, science, and everyday life for human (Gabora and Kaufman, 2010), it is non-trivial if models could produce creative content.…”