Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis

Huang, Faliang; Li, Xuelong; Yuan, Changan; Zhang, Shichao; Zhang, Jilian; Qiao, Shaojie

doi:10.1109/tnnls.2021.3056664

Cited by 98 publications

(40 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [60], the authors introduced a transformer-based model for emotion detection that encodes representation from a transformer and applies deep embedding to improve the quality of tweets. In [61], the authors introduced an attention-based deep method using two independent layers. By having to consider temporal information flow in two directions, it will retrieve both past and future contexts.…”

Section: Related Workmentioning

confidence: 99%

Weighted Joint Sentiment-Topic Model for Sentiment Analysis Compared to ALGA: Adaptive Lexicon Learning Using Genetic Algorithm

Osmani

Mohasefi

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Latent Dirichlet Allocation (LDA) is an approach to unsupervised learning that aims to investigate the semantics among words in a document as well as the influence of a subject on a word. As an LDA-based model, Joint Sentiment-Topic (JST) examines the impact of topics and emotions on words. The emotion parameter is insufficient, and additional parameters may play valuable roles in achieving better performance. In this study, two new topic models, Weighted Joint Sentiment-Topic (WJST) and Weighted Joint Sentiment-Topic 1 (WJST1), have been presented to extend and improve JST through two new parameters that can generate a sentiment dictionary. In the proposed methods, each word in a document affects its neighbors, and different words in the document may be affected simultaneously by several neighbor words. Therefore, proposed models consider the effect of words on each other, which, from our view, is an important factor and can increase the performance of baseline methods. Regarding evaluation results, the new parameters have an immense effect on model accuracy. While not requiring labeled data, the proposed methods are more accurate than discriminative models such as SVM and logistic regression in accordance with evaluation results. The proposed methods are simple with a low number of parameters. While providing a broad perception of connections between different words in documents of a single collection (single-domain) or multiple collections (multidomain), the proposed methods have prepared solutions for two different situations (single-domain and multidomain). WJST is suitable for multidomain datasets, and WJST1 is a version of WJST which is suitable for single-domain datasets. While being able to detect emotion at the level of the document, the proposed models improve the evaluation outcomes of the baseline approaches. Thirteen datasets with different sizes have been used in implementations. In this study, perplexity, opinion mining at the level of the document, and topic_coherency are employed for assessment. Also, a statistical test called Friedman test is used to check whether the results of the proposed models are statistically different from the results of other algorithms. As can be seen from results, the accuracy of proposed methods is above 80% for most of the datasets. WJST1 achieves the highest accuracy on Movie dataset with 97 percent, and WJST achieves the highest accuracy on Electronic dataset with 86 percent. The proposed models obtain better results compared to Adaptive Lexicon learning using Genetic Algorithm (ALGA), which employs an evolutionary approach to make an emotion dictionary. Results show that the proposed methods perform better with different topic number settings, especially for WJST1 with 97% accuracy at |Z| = 5 on the Movie dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Weighted Joint Sentiment-Topic Model for Sentiment Analysis Compared to ALGA: Adaptive Lexicon Learning Using Genetic Algorithm

Osmani

Mohasefi

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…The most important concept of LSTM is the cell state. The cell state is the mechanism to transfer information throughout the network [62], the cell state is the memory of the network, which allows it to remember or forget information. This information is added or removed by three gates, namely the input, output, and forget gates, defined by Equations (1)-( 3)), respectively.…”

Section: Long-short-term Memory (Lstm)mentioning

confidence: 99%

Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification

2021

View full text Add to dashboard Cite

The traditional way to address the problem of sentiment classification is based on machine learning techniques; however, these models are not able to grasp all the richness of the text that comes from different social media, personal web pages, blogs, etc., ignoring the semantic of the text. Knowledge graphs give a way to extract structured knowledge from images and texts in order to facilitate their semantic analysis. This work proposes a new hybrid approach for Sentiment Analysis based on Knowledge Graphs and Deep Learning techniques to identify the sentiment polarity (positive or negative) in short documents, such as posts on Twitter. In this proposal, tweets are represented as graphs; then, graph similarity metrics and a Deep Learning classification algorithm are applied to produce sentiment predictions. This approach facilitates the traceability and interpretability of the classification results, thanks to the integration of the Local Interpretable Model-agnostic Explanations (LIME) model at the end of the pipeline. LIME allows raising trust in predictive models, since the model is not a black box anymore. Uncovering the black box allows understanding and interpreting how the network could distinguish between sentiment polarities. Each phase of the proposed approach conformed by pre-processing, graph construction, dimensionality reduction, graph similarity, sentiment prediction, and interpretability steps is described. The proposal is compared with character n-gram embeddings-based Deep Learning models to perform Sentiment Analysis. Results show that the proposal is able to outperforms classical n-gram models, with a recall up to 89% and F1-score of 88%.

show abstract

“…Due to remembering information for a long time and removing the vanishing gradient problem of RNN, LSTM has appeared to be an effective model in solving problems with sequential data containing long-term dependencies. Some of the examples of LSTM applications are speech recognition [36], machine translation [37], time series forecasting [38,39], and sentiment analysis [40]. Hochreiter and Schmidhuber [41] in 1997 first proposed the LSTM model in which each LSTM unit contains only input and output gates.…”

Section: Long Short-term Memory (Lstm)mentioning

confidence: 99%

Comparative Study of Univariate and Multivariate Long Short-Term Memory for Very Short-Term Forecasting of Global Horizontal Irradiance

et al. 2021

View full text Add to dashboard Cite

Accurate global horizontal irradiance (GHI) forecasting is crucial for efficient management and forecasting of the output power of photovoltaic power plants. However, developing a reliable GHI forecasting model is challenging because GHI varies over time, and its variation is affected by changes in weather patterns. Recently, the long short-term memory (LSTM) deep learning network has become a powerful tool for modeling complex time series problems. This work aims to develop and compare univariate and several multivariate LSTM models that can predict GHI in Guntur, India on a very short-term basis. To build the multivariate time series models, we considered all possible combinations of temperature, humidity, and wind direction variables along with GHI as inputs and developed seven multivariate models, while in the univariate model, we considered only GHI variability. We collected the meteorological data for Guntur from 1 January 2016 to 31 December 2016 and built 12 datasets, each containing variability of GHI, temperature, humidity, and wind direction of a month. We then constructed the models, each of which measures up to 2 h ahead of forecasting of GHI. Finally, to measure the symmetry among the models, we evaluated the performances of the prediction models using root mean square error (RMSE) and mean absolute error (MAE). The results indicate that, compared to the univariate method, each multivariate LSTM performs better in the very short-term GHI prediction task. Moreover, among the multivariate LSTM models, the model that incorporates the temperature variable with GHI as input has outweighed others, achieving average RMSE values 0.74 W/m2–1.5 W/m2.

show abstract

Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis

Cited by 98 publications

References 62 publications

Weighted Joint Sentiment-Topic Model for Sentiment Analysis Compared to ALGA: Adaptive Lexicon Learning Using Genetic Algorithm

Weighted Joint Sentiment-Topic Model for Sentiment Analysis Compared to ALGA: Adaptive Lexicon Learning Using Genetic Algorithm

Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification

Comparative Study of Univariate and Multivariate Long Short-Term Memory for Very Short-Term Forecasting of Global Horizontal Irradiance

Contact Info

Product

Resources

About