Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition

Xie, Yubo; Li, Junze; Pu, Pearl

doi:10.18653/v1/2021.acl-short.6

Cited by 4 publications

(1 citation statement)

References 24 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Weller and Seppi (2019) demonstrated that the BERT-based classifier achieves superior results in humor recognition task on several existing datasets, and also proposed a new dataset composed of humorous Reddit posts. There are studies leveraging the setup-punchline structure of jokes for humor detection task using BERT senetence embeddings (Annamoradnejad and Zoghi, 2020) or GPT-2 (Xie et al, 2021). Peyrard et al (2021) applied various models -three transformer models (BERT, distilBERT, and RoBERTa), fastText-based representation, GPT-2, and LSTM, -to a collection of aligned humorous/non-humorous sentence pairs.…”

Section: Related Workmentioning

confidence: 99%

You Told Me That Joke Twice: A Systematic Investigation of Transferability and Robustness of Humor Detection Models

Baranov,

Kniazhevsky,

Braslavski

2023

Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

In this study, we focus on automatic humor detection, a highly relevant task for conversational AI. To date, there are several English datasets for this task, but little research on how models trained on them generalize and behave in the wild. To fill this gap, we carefully analyze existing datasets, train RoBERTabased and Naïve Bayes classifiers on each of them, and test on the rest. Training and testing on the same dataset yields good results, but the transferability of the models varies widely. Models trained on datasets with jokes from different sources show better transferability, while the amount of training data has a smaller impact. The behavior of the models on out-of-domain data is unstable, suggesting that some of the models overfit, while others learn non-specific humor characteristics. An adversarial attack shows that models trained on pun datasets are less robust. We also evaluate the sense of humor of the chatGPT and Flan-UL2 models in a zero-shot scenario. The LLMs demonstrate competitive results on humor datasets and a more stable behavior on out-of-domain data. We believe that the obtained results will facilitate the development of new datasets and evaluation methodologies in the field of computational humor. We've made all the data from the study and the trained models publicly available: https://github.com/ Humor-Research/Humor-detection.Tell me which are funny, which are not -and which get a giggle first time but are cold pancakes without honey to hear twice.

show abstract

Section: Related Workmentioning

confidence: 99%