SemEval-2018 Task 3: Irony Detection in English Tweets

Abstract

This paper presents the first shared task on irony detection: given a tweet, automatic natural language processing systems should determine whether the tweet is ironic (Task A) and which type of irony (if any) is expressed (Task B). The ironic tweets were collected using irony-related hashtags (i.e. #irony, #sar-casm, #not) and were subsequently manually annotated to minimise the amount of noise in the corpus. Prior to distributing the data, hashtags that were used to collect the tweets were removed from the corpus. For both tasks, a training corpus of 3,834 tweets was provided, as well as a test set containing 784 tweets. Our shared tasks received submissions from 43 teams for the binary classification Task A and from 31 teams for the multiclass Task B. The highest classification scores obtained for both subtasks are respectively F 1 = 0.71 and F 1 = 0.51 and demonstrate that fine-grained irony classification is much more challenging than binary irony detection.

INAOE-UPV at SemEval-2018 Task 3: An Ensemble Approach for Irony Detection in Twitter

“…This year, as part of SemEval-2018 the Task 3 on Irony detection in English tweets (Van Hee et al, 2018), was dedicated to the identification of ironic content in Twitter. The task is composed by two subtasks: Task A.…”

“…In this paper we describe our participation to the SemEval-2018 Task 3: Irony detection in English tweets (Van Hee et al, 2018). The INAOE-UPV system explores the use of an ensemble approach that considers different combinations of three methods.…”

THU_NGN at SemEval-2018 Task 3: Tweet Irony Detection with Densely connected LSTM and Multi-task Learning

“…To fill this gap, the SemEval-2018 task 3 1 aims to detect irony of tweets without explicit irony hashtags (Van Hee et al, 2018). The subtask A is aimed to determine whether a tweet is ironic.…”

WLV at SemEval-2018 Task 3: Dissecting Tweets in Search of Irony

“…As part of the 12th workshop on semantic evaluation (SemEval-2018), Shared Task 3 defines two subtasks with regards to irony detection in English tweets (Van Hee et al, 2018). Subtask A involves binary classification.…”