A Convolutional Stacked Bidirectional LSTM with a Multiplicative Attention Mechanism for Aspect Category and Sentiment Detection

J, Ashok Kumar; Trueman, Tina Esther; Wang, Zhaoxia

doi:10.1007/s12559-021-09948-0

Cited by 35 publications

(6 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As explored in detail in the findings, the proposed model with its ensemble teacher-student framework has considerably outperformed the baseline SVM-based models ( Al-Smadi et al, 2016 ; Pontiki et al, 2016 ) by 27.9% in the micro F1. Compared to the RNN, LSTM, and CNN implementations reported in the related literature ( Tamchyna & Veselovsk’a, 2016 ; Bensoltane & Zaki, 2021 ; Kumar, Trueman & Cambria, 2021 ), our model has demonstrated a significant improvement ranging from 16% to 20% in micro F1 metric. The proposed model has as well performed better than the highest performing DL model reported in the literature (BERT embedding with BiGRU) by 2.7% in the ACD task.…”

Section: Discussionmentioning

confidence: 78%

“…The BERT embeddings achieved the highest F1 of 63.6 and 65.5 when added to the GRU and BiGRU models respectively. Kumar, Trueman & Cambria (2021) have proposed a convolutional stacked bidirectional LSTM with a multiplicative attention mechanism for aspect category detection and sentiment analysis. When evaluated, the CNN and stacked BiLSTM Multi Attn model achieved the same weighted F1 (=0.52) on the SemEval 2015 dataset, however it achieved a higher weighted F1 of 0.60 on the SemEval 2016 dataset than the CNN stacked BiLSTM Attn which scored 0.58.…”

Section: Review Of the Related Literaturementioning

confidence: 99%

See 1 more Smart Citation

A semi supervised approach to Arabic aspect category detection using Bert and teacher-student model

Almasri

Al-Malki

Alotaibi

2023

PeerJ Computer Science

View full text Add to dashboard Cite

Aspect-based sentiment analysis tasks are well researched in English. However, we find such research lacking in the context of the Arabic language, especially with reference to aspect category detection. Most of this research is focusing on supervised machine learning methods that require the use of large, labeled datasets. Therefore, the aim of this research is to implement a semi-supervised self-training approach which utilizes a noisy student framework to enhance the capability of a deep learning model, AraBERT v02. The objective is to perform aspect category detection on both the SemEval 2016 hotel review dataset and the Hotel Arabic-Reviews Dataset (HARD) 2016. The four-step framework firstly entails developing a teacher model that is trained on the aspect categories of the SemEval 2016 labeled dataset. Secondly, it generates pseudo labels for the unlabeled HARD dataset based on the teacher model. Thirdly, it creates a noisy student model that is trained on the combined datasets (∼1 million sentences). The aim is to minimize the combined cross entropy loss. Fourthly, an ensembling of both teacher and student models is carried out to enhance the performance of AraBERT. Findings indicate that the ensembled teacher-student model demonstrates a 0.3% improvement in its micro F1 over the initial noisy student implementation, both in predicting the Aspect Categories in the combined datasets. However, it has achieved a 1% increase over the micro F1 of the teacher model. These results outperform both baselines and other deep learning models discussed in the related literature.

show abstract

Section: Discussionmentioning

confidence: 78%

Section: Review Of the Related Literaturementioning

confidence: 99%

A semi supervised approach to Arabic aspect category detection using Bert and teacher-student model

Almasri

Al-Malki

Alotaibi

2023

PeerJ Computer Science

View full text Add to dashboard Cite

show abstract

“…Then, models with lower AIC values are preferred due to their improved balance between fit and complexity. We compare the HDRB model regarding recall, precision, and f1-score criteria with the results of Trueman et al [32] The authors proposed the hybrid model, a convolutional stacked BiLSTM with a multiplicative/single attention mechanism. The convolutional layers capture local patterns and features from the input data, while the bidirectional LSTM allows the model to consider past and future contexts.…”

Section: ) Hdrb Model Resultsmentioning

confidence: 99%

Aspect-Based Sentiment Analysis of Twitter Influencers to Predict the Trend of Cryptocurrencies Based on Hybrid Deep Transfer Learning Models

Jahanbin,

Chahooki

2023

IEEE Access

View full text Add to dashboard Cite

With the expansion of social networks, sentiment analysis has become one of the hot topics in machine learning. However, in traditional sentiment analysis, the text is considered of a general nature and ignores the different aspects that may exist in the text. This paper presents a hybrid model of transfer deep learning methods for the aspect-oriented sentiment analysis of influencers' tweets to predict the trend of cryptocurrencies. In the first model, different aspects of tweets are extracted using the Concept Latent Dirichlet Allocation (Concept-LDA). Then, by using the pre-trained RoBERTa network and combining it with the Bidirectional Gated Recurrent Unit (BiGRU) deep learning network and attention layer, sentiments of different aspects of tweets are determined. In the following, the price trend of seven cryptocurrencies, Bitcoin, Ethereum, Binance, Ripple, Dogecoin, Cardano, and Solana, is determined using the historical price and the polarity of tweets with BiGRU combined deep neural network and the attention layer. Also, we used the gridsearch method to select dropout hyper-parameters, learning rate, and the number of GRU units, and the Akaike Information Criterion (AIC) criterion confirmed the results of this proposed combination. The results show that the proposed model in the aspect-based sentiment analysis section has been able to achieve 5.94% accuracy and 9.9% improvement in the f1-score on the SemEval 2015 dataset and 2.61% improvement on the SemEval 2016 dataset in f1-score compared to the state-of-arts. Also, the results of predicting the price trend of cryptocurrencies show that the proposed model has correctly recognized the price trend in the next five days in 77% of cases according to the ROC-AUC criterion.INDEX TERMS Aspect Based sentiment analysis, Prediction trend price, Cryptocurrencies, Pre-trained networks, Hybrid deep learning models.

show abstract

“…This model achieved the best performance results on SemEval-2017 [13]. The study by [17] developed a convolutional stacked BiLSTM with a multiplicative attention mechanism for the purpose of detecting aspect category and sentiment polarity. The evaluation of the model was carried out as a multiclass classification.…”

Section: English Sentiment Analysismentioning

confidence: 99%

Arabic sentiment analysis using dependency-based rules and deep neural networks

Diwali

Dashtipour

Saeedi

et al. 2022

Applied Soft Computing

Self Cite

View full text Add to dashboard Cite

A Convolutional Stacked Bidirectional LSTM with a Multiplicative Attention Mechanism for Aspect Category and Sentiment Detection

Cited by 35 publications

References 56 publications

A semi supervised approach to Arabic aspect category detection using Bert and teacher-student model

A semi supervised approach to Arabic aspect category detection using Bert and teacher-student model

Aspect-Based Sentiment Analysis of Twitter Influencers to Predict the Trend of Cryptocurrencies Based on Hybrid Deep Transfer Learning Models

Arabic sentiment analysis using dependency-based rules and deep neural networks

Contact Info

Product

Resources

About