¡Qué maravilla! Multimodal Sarcasm Detection in Spanish: a Dataset and a Baseline

Abstract

We construct the first ever multimodal sarcasm dataset for Spanish. The audiovisual dataset consists of sarcasm annotated text that is aligned with video and audio. The dataset represents two varieties of Spanish, a Latin American variety and a Peninsular Spanish variety, which ensures a wider dialectal coverage for this global language. We present several models for sarcasm detection that will serve as baselines in the future research. Our results show that results with text only (89%) are worse than when combining text with audio (91.9%). Finally, the best results are obtained when combining all the modalities: text, audio and video (93.1%).

Comment-Aware Multi-Modal Heterogeneous Pre-Training for Humor Detection in Short-Form Videos

“…Hasan et al [15] and Hasan et al [14] aim to detect the punchline humor label by multi-modal information which involves text (transcriptions), audio, and video extracted from TED talks in both context and punchline. Moreover, many existing studies [2,6,7,36] detect the utterance humor in dialogue scenarios on datasets produced by TV shows from different cultures.…”