Multi-Label Emotion Classification on Code-Mixed Text: Data and Methods

Abstract

The multi-label emotion classification task aims to identify all possible emotions in a written text that best represent the author's mental state. In recent years, multi-label emotion classification attracted the attention of researchers due to its potential applications in e-learning, health care, marketing, etc. There is a need for standard benchmark corpora to develop and evaluate multi-label emotion classification methods. The majority of benchmark corpora were developed for the English language (monolingual corpora) using tweets. However, the multi-label emotion classification problem is not explored for codemixed text, for example, English and Roman Urdu, although the code-mixed text is widely used in Facebook posts/comments, tweets, SMS messages, particularly by the South Asian community. For filling this gap, this study presents a large benchmark corpus for the multi-label emotion classification task, which comprises 11,914 code-mixed (English and Roman Urdu) SMS messages. Each code-mixed (English and Roman Urdu) SMS message manually annotated using a set of 12 emotions, including anger, anticipation, disgust, fear, joy, love, optimism, pessimism, sadness, surprise, trust, and neutral (no emotion). As a secondary contribution, we applied and compared state-of-the-art classical machine learning (content-based methods -three word n-gram features and eight character n-gram features), deep learning (CNN, RNN, Bi-RNN, GRU, Bi-GRU, LSTM, and Bi-LSTM), and transfer learning-based methods (BERT and XLNet) on our proposed corpus. After our extensive experimentation, the best results were obtained using state-of-the-art classical machine learning methods on word uni-gram (Micro Precision = 0.67, Micro Recall = 0.54, Micro F 1 = 0.67) with a combination of OVR multi-label and SVC single-label machine learning algorithms. Our proposed corpus is free and publicly available for research purposes to foster research in an under-resourced language (Roman Urdu).

Mental Illness Classification on Social Media Texts using Deep Learning and Transfer Learning

“…The BERT model provided state-of-the-art performance over different NLP tasks without any critical task-specific design changes [22,13]. [5] used the BERT model for classification task on the multilabel text, which is trained by Google [13]. The model was developed to enable transfer learning, which is why it went through a pre-training procedure that included utilizing both the BookCorpus and the English Wikipedia to help the model learn English.…”

Understanding Emotions: A PoliEMO Datasetand Multi-label Classification in Indian Elections

“…CBET [42] and BMET [22] corpus are also English language corpus for multi-label classification. Ameer [7] developed a multi-label dataset for English and Urdu language SMS for multi-label emotions detection. There are a few more datasets built for other languages [37,46].…”

“…On the contrary, people can have multiple emotions simultaneously depending upon the thought and situation. Work presents the classification of social media text into multiple emotional states [5][6][7]51] such as joy, anger, sadness, and shameful. Hence, multi-label emotion classification allows user text to classify into more than one label to understand the user's emotions completely.…”

The Evaluation on the Credit Risk of Enterprises with the CNN-LSTM-ATT Model

“…With the development of the economy and the development of neural networks, it is necessary to combine neural networks and risk evaluation issues. At the same time, the model combining CNN and LSTM has been applied to text classi cation [21][22][23][24], image recognition [25][26][27][28], emotion analysis [21,29,30], and other elds. Based on the deep learning method of CNN and LSTM, this paper aims at studying the problem of the enterprise's credit score by using enterprise's behavior data.…”