Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text

Tang, Tiancheng; Tang, Xinhuai; Yuan, Tianyi

doi:10.1109/access.2020.3030468

Cited by 41 publications

(13 citation statements)

References 17 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Contextualized Text Embeddings Contextualized embeddings have been intensively studied in NLP. Self-supervised large-scale pretraining facilitates the learning of semantic and contextual information and benefits various downstream applications such as text classification (Sun et al, 2019), sentiment analysis (Tang et al, 2020;Song et al, 2020) and relation extraction (Alt et al, 2019). There are also many domain-specific variants of pretrained contextualized text embeddings.…”

Section: Related Workmentioning

confidence: 99%

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

Ji¹,

Zhang²,

Ansari³

et al. 2021

Preprint

View full text Add to dashboard Cite

Mental health is a critical issue in modern society, and mental disorders could sometimes turn to suicidal ideation without adequate treatment. Early detection of mental disorders and suicidal ideation from social content provides a potential way for effective social intervention. Recent advances in pretrained contextualized language representations have promoted the development of several domain-specific pretrained models and facilitated several downstream applications. However, there are no existing pretrained language models for mental healthcare. This paper trains and release two pretrained masked language models, i.e., MentalBERT and MentalRoBERTa, to benefit machine learning for the mental healthcare research community. Besides, we evaluate our trained domain-specific models and several variants of pretrained language models on several mental disorder detection benchmarks and demonstrate that language representations pretrained in the target domain improve the performance of mental health detection tasks.

show abstract

Section: Related Workmentioning

confidence: 99%

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

Ji¹,

Zhang²,

Ansari³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In recent years, there have been various attempts in emotion prediction that have used BERT as the base model or in ensemble with some other approach. [44] investigated the problem of emotion classification in code-switched documents using BERT. The task of emotion classification and constructing emotion lexicon was combined by [45] that improved emotion classification results on the Twitter dataset.…”

Section: Related Workmentioning

confidence: 99%

Fine-Grained Emotion Prediction by Modeling Emotion Definitions

Singh¹,

Brahma²,

Rai³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a new framework for fine-grained emotion prediction in the text through emotion definition modeling. Our approach involves a multi-task learning framework that models definitions of emotions as an auxiliary task while being trained on the primary task of emotion prediction. We model definitions using masked language modeling and class definition prediction tasks. Our models outperform existing state-of-the-art for fine-grained emotion dataset GoEmotions. We further show that this trained model can be used for transfer learning on other benchmark datasets in emotion prediction with varying emotion label sets, domains, and sizes. The proposed models outperform the baselines on transfer learning experiments demonstrating the generalization capability of the models.

show abstract

“…It also focuses on the automatic uncovering of the considered attitudes [10]. Understanding the user's perspective is significant for different applications like marketing analysis, product feedback, political campaigns, product reviews, and public relations [36]. The sentiment analysis has also been applied in complex problems like security threats such as monitoring conversations regarding terrorism [45].…”

Section: Introductionmentioning

confidence: 99%

SenDemonNet: sentiment analysis for demonetization tweets using heuristic deep neural network

Kayıkçı

2022

Multimed Tools Appl

View full text Add to dashboard Cite

Sentiment analysis is one of the efficient models for extracting opinion mining with identification and classification from unstructured text data such as product reviews or microblogs. It is used to gain feedback from political campaigns, brand reviews, marketing analysis, and customers. The sentiment analysis on Twitter data is a recent research field in the natural processing. The dataset is gathered from the "Twitter" package in R along with Twitter API. The main intent of this paper is to understand the public opinion on the recently implemented demonetization policy using the proposed SenDemonNet. Initially, the tweet preprocessing was done, which is intended for cleaning the text data. Then, the feature extraction is performed by Bag of n-grams, TF-IDF, and the word2vec algorithm. The main objective of this work is a weighted feature selection that is developed by the hybrid Forest-Whale Optimization Algorithm (F-WOA) to get the best classification outcome. With these features, the Heuristic Deep Neural Network (HDNN) is adopted for classification, where the proposed FOA and WOA tune the parameter of DNN for reaching the maximum accuracy rate. From the statistical analysis, the performance of the designed F-WOA-DNN is 1.8%, 1.9%, 1.86%, and 2% enhanced than PSO-DNN, GWO-DNN, WOA-DNN, FOA-DNN, SVM, CNN, LSTM, and DNN respectively. Extensive experimental results show that SenDemonNet outperforms its competitors, producing an impressive increase in the classification accuracy on the benchmark dataset.

show abstract

Fine-Tuning BERT for Multi-Label Sentiment Analysis in Unbalanced Code-Switching Text

Cited by 41 publications

References 17 publications

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare

Fine-Grained Emotion Prediction by Modeling Emotion Definitions

SenDemonNet: sentiment analysis for demonetization tweets using heuristic deep neural network

Contact Info

Product

Resources

About