Influence of Weak Labels for Emotion Recognition of Tweets

Abstract

Abstract.Research on emotion recognition of tweets focuses on feature engineering or algorithm design, while dataset labels are barely questioned. Datasets of tweets are often labelled manually or via crowdsourcing, which results in strong labels. These methods are time intensive and can be expensive. Alternatively, tweet hashtags can be used as free, inexpensive weak labels. This paper investigates the impact of using weak labels compared to strong labels. The study uses two label sets for a corpus of tweets. The weakly annotated label set is created employing the hashtags of the tweets, while the strong label set is created by the use of crowdsourcing. Both label sets are used separately as input for five classification algorithms to determine the classification performance of the weak labels. The results indicate only a 9.25% decrease in f1-score when using weak labels. This performance decrease does not outweigh the benefits of having free labels.

Multi-thread hierarchical deep model for context-aware sentiment analysis

“…In this line, researchers use sentimental hashtags and emoticons to label tweets as positive or negative. Mohammad [43] conducted two experiments which showed that the hashtag-based labels were consistent with the human annotations [7], and Janssens et al [45] compared valence of hashtag labels and human labels and indicated a high degree of agreement between them. Based on the previous research works, we used #happy, #excellent, #happiness, #sadness, #sad and #frustrated to collect target tweets using the Twitter official application programming interface (API) in 14 April 2019.…”

“…It has been reported that Twitter is the main social media data source for sentiment analysis [4] and context-aware sentiment analysis [8]. For annotating data, we use a popular annotation approach for Twitter sentiment analysis called distant supervision [9,25,[43][44][45][46], also known as indirect crowdsourcing [7]. In this line, researchers use sentimental hashtags and emoticons to label tweets as positive or negative.…”