Attention Models for Sentiment Analysis Using Objectivity and Subjectivity Word Vectors

Lee, Wing Shum; Ng, Hu; Yap, Timothy Tzen Vun; Ho, Chiung Ching; Goh, Vik Tor; Tong, Hau-Lee

doi:10.1007/978-981-33-4069-5_5

“…This paper adopted attention-weighted features model is called Long Short Term Memory with Attention (LSTM-ATT) 4 with intention to improve the sentiment performance. These features at the input level to the neural network and go through a few dense layers to flatten the output.…”

Section: Modelsmentioning

confidence: 99%

“…This paper adopted attention segment 3 to a neural network, LSTM, by creating attention-weighted features, namely Long Short Term Memory with Attention (LSTM-ATT) 4 to create attention-weighted features. It aims to introduce these features at the input level to the neural network, so that the performance of sentiment can be increased.…”

Section: Introductionmentioning

confidence: 99%

Modelling sentiments based on objectivity and subjectivity with self-attention mechanisms

Ng

¹

,

Chia

²

,

Yap

³

et al. 2021

F1000Res

Self Cite

0

View full text Add to dashboard Cite

Background: The proliferation of digital commerce has allowed merchants to reach out to a wider customer base, prompting a study of customer reviews to gauge service and product quality through sentiment analysis. Sentiment analysis can be enhanced through subjectivity and objectivity classification with attention mechanisms. Methods: This research includes input corpora of contrasting levels of subjectivity and objectivity from different databases to perform sentiment analysis on user reviews, incorporating attention mechanisms at the aspect level. Three large corpora are chosen as the subjectivity and objectivity datasets, the Shopee user review dataset (ShopeeRD) for subjectivity, together with the Wikipedia English dataset (Wiki-en) and Internet Movie Database (IMDb) for objectivity. Word embeddings are created using Word2Vec with Skip-Gram. Then, a bidirectional LSTM with an attention layer (LSTM-ATT) imposed on word vectors. The performance of the model is evaluated and benchmarked against classification models of Logistics Regression (LR) and Linear SVC (L-SVC). Three models are trained with subjectivity (70% of ShopeeRD) and the objectivity (Wiki-en) embeddings, with ten-fold cross-validation. Next, the three models are evaluated against two datasets (IMDb and 20% of ShopeeRD). The experiments are based on benchmark comparisons, embedding comparison and model comparison with 70-10-20 train-validation-test splits. Data augmentation using AUG-BERT is performed and selected models incorporating AUG-BERT, are compared. Results: L-SVC scored the highest accuracy with 56.9% for objective embeddings (Wiki-en) while the LSTM-ATT scored 69.0% on subjective embeddings (ShopeeRD). Improved performances were observed with data augmentation using AUG-BERT, where the LSTM-ATT+AUG-BERT model scored the highest accuracy at 60.0% for objective embeddings and 70.0% for subjective embeddings, compared to 57% (objective) and 69% (subjective) for L-SVC+AUG-BERT, and 56% (objective) and 68% (subjective) for L-SVC. Conclusions: Utilizing attention layers with subjectivity and objectivity notions has shown improvement to the accuracy of sentiment analysis models.

show abstract

“…On the other hand, a non-fact statement is subjective and usually related to an individual's sentiments, personal beliefs, opinion, perspective, feelings or thoughts. This paper adopted attention segment 3 to a neural network, LSTM, by creating attention-weighted features, namely Long Short Term Memory with Attention (LSTM-ATT) 4 to create attention-weighted features. It aims to introduce these features at the input level to the neural network, so that the performance of sentiment can be increased.…”

Section: Introductionmentioning

confidence: 99%

Modelling sentiments based on objectivity and subjectivity with self-attention mechanisms

Ng

¹

,

Chia

²

,

Yap

³

et al. 2022

Self Cite

View full text Add to dashboard Cite

Background: The proliferation of digital commerce has allowed merchants to reach out to a wider customer base, prompting a study of customer reviews to gauge service and product quality through sentiment analysis. Sentiment analysis can be enhanced through subjectivity and objectivity classification with attention mechanisms. Methods: This research includes input corpora of contrasting levels of subjectivity and objectivity from different databases to perform sentiment analysis on user reviews, incorporating attention mechanisms at the aspect level. Three large corpora are chosen as the subjectivity and objectivity datasets, the Shopee user review dataset (ShopeeRD) for subjectivity, together with the Wikipedia English dataset (Wiki-en) and Internet Movie Database (IMDb) for objectivity. Word embeddings are created using Word2Vec with Skip-Gram. Then, a bidirectional LSTM with an attention layer (LSTM-ATT) imposed on word vectors. The performance of the model is evaluated and benchmarked against classification models of Logistics Regression (LR) and Linear SVC (L-SVC). Three models are trained with subjectivity (70% of ShopeeRD) and the objectivity (Wiki-en) embeddings, with ten-fold cross-validation. Next, the three models are evaluated against two datasets (IMDb and 20% of ShopeeRD). The experiments are based on benchmark comparisons, embedding comparison and model comparison with 70-10-20 train-validation-test splits. Data augmentation using AUG-BERT is performed and selected models incorporating AUG-BERT, are compared. Results: L-SVC scored the highest accuracy with 56.9% for objective embeddings (Wiki-en) while the LSTM-ATT scored 69.0% on subjective embeddings (ShopeeRD). Improved performances were observed with data augmentation using AUG-BERT, where the LSTM-ATT+AUG-BERT model scored the highest accuracy at 60.0% for objective embeddings and 70.0% for subjective embeddings, compared to 57% (objective) and 69% (subjective) for L-SVC+AUG-BERT, and 56% (objective) and 68% (subjective) for L-SVC. Conclusions: Utilizing attention layers with subjectivity and objectivity notions has shown improvement to the accuracy of sentiment analysis models.

show abstract

“…The Language Processing Algorithm is utilized for extracting Features such as Phrases, Word Frequency, Parts of Speech Tags, and Opinion Words [7]. But the Supervised Machine Learning (ML) algorithm learns the Polarity (Positive, Negative, or Neutral) of the Reviews from a data that is primarily categorized by a human [8]. Then, Scores are allocated to the Opinion Word according to Positive or Negative the words contained in the Dictionary [9].…”

Section: Introductionmentioning

confidence: 99%

Twitter Text Sentiment Analysis of Amazon Unlocked Mobile Reviews Using Supervised Learning Techniques

Bharathi¹,

R.²,

R.³

2022

INDJCSE

1

0

View full text Add to dashboard Cite

Due to the drastic increase in usage of Internet and e-commerce, online shopping becomes very common in day to day lives. One of the popular e-commerce websites is Amazon, which allows users to post Reviews and Ratings regarding the purchased products. It finds useful for other customers and companies to improve the product quality. Since the manual extraction of sentiments from User Product Reviews is a tedious process, accurate and Automated Sentiment Analysis (SA) tools are essential. This research work aims to examine the performance of different Machine Learning (ML) models in analyzing the sentiments of Amazon Product Reviews. Initially, the Product Reviews are pre-processed in different ways to transform the data into a useful format. Besides, Term Frequency -Inverse Document Frequency (TF-IDF) vectorizer is used to derive Feature Embeddings. Finally, three Machine Learning (ML) models namely Gaussian Naive Bayes (GNB), Logistic Regression (LR), and Support Vector Machines (SVM) are used for Sentiment Analysis (SA). The Performance Validation of the Machine Learning (ML) models is performed using Benchmark Dataset from Kaggle Repository. The Experimental Results reported that the Support Vector Machine (SVM) model has resulted in better Performance over the other Machine Learning (ML) models interms of Different Measures.

show abstract

Attention Models for Sentiment Analysis Using Objectivity and Subjectivity Word Vectors

Cited by 3 publications

References 14 publications

Modelling sentiments based on objectivity and subjectivity with self-attention mechanisms

Modelling sentiments based on objectivity and subjectivity with self-attention mechanisms

Modelling sentiments based on objectivity and subjectivity with self-attention mechanisms

Twitter Text Sentiment Analysis of Amazon Unlocked Mobile Reviews Using Supervised Learning Techniques

Contact Info

Product

Resources

About