A heterogeneous stacking ensemble based sentiment analysis framework using multiple word embeddings

Abstract

Word embedding techniques have been proposed in the literature to analyze and determine the sentiments expressed in various textual documents such as social media posts, online product reviews, and so forth. However, it is difficult to capture the entire gamut of intricate inter-dependencies among words in the textual documents using a specific word embedding technique. In this article, we aim to address this issue by proposing a computation-efficient stacking ensemble based sentiment analysis framework using multiple word embeddings. The proposed framework uses a combination of three distinct word embeddings generated by three different state of the art word embedding techniques, namely, Word2Vec, GloVe, and BERT for performing the sentiment analysis task. It uses an explicitly trained Word2Vec model to generate the first set of 200-dimensional word embedding. Similarly, pre-trained GloVe and BERT models are used to generate the other two sets of 200-dimensional and a 768-dimensional word embeddings, respectively. These three distinct word embedding sets are then used to train a heterogeneous stacking ensemble based classifier model comprising LSTM, GRU, and Bi-GRU based base-level classifiers, and a LSTM based meta-level classifier. Experimental results on four different datasets, 530

Effect of word embedding vector dimensionality on sentiment analysis through short and long texts

“…Alexandridis et al [27] used various language models to represent social media texts and Greek language text classifiers, using word embedding implemented by the GloVe model, to detect the polarity of opinions expressed on social media. The GloVe model has also been used in sentiment analysis models, often associated with a recurrent neural network module like long sort-term memory (LSTM) or GRU [6], [28], [29].…”

Heterogeneous Ensemble Deep Learning Model for Enhanced Arabic Sentiment Analysis

“…In most cases, Ensemble learning methods can be in the form of three popular ones, namely bagging [ 17 ], boosting [ 18 ], and stacking [ 19 ]. Many Researcher activities on ensemble learning centered around homogeneous ensembles, even though heterogeneous ensembles could prove more efficient in case of combining pre-trained models that are often readily available such as [ 20 , 21 ].…”

Improving word embeddings in Portuguese: increasing accuracy while reducing the size of the corpus

“…Different distributional semantics models have been developed to generate embeddings, and these have proved to adequately capture the semantic properties of words, as long as sufficiently large corpora is used ( Joulin et al, 2016 ; Mikolov et al, 2013 ). Several application scenarios have been tested including: classification of twitter streams ( Khatua, Khatua & Cambria, 2019 ; Zhang & Luo, 2019 ), plagiarism detection ( Tien et al, 2019 ), opinion mining on social networks ( Nguyen & Le Nguyen, 2018 ; Rida-E-Fatima et al, 2019 ), recommendation systems ( Chamberlain et al, 2020 ; Baek & Chung, 2021 ), mapping of scientific domain keywords ( Hu et al, 2019 ), tracking emerging scientific keywords ( Dridi et al, 2019 ), optimization of queries for Information Retrieval ( Roy et al, 2019 ; Hofstätter et al, 2019 ) or sentiment analysis ( Santosh Kumar, Yadav & Dhavale, 2021 ; Subba & Kumari, 2022 ). However, the great majority of such models has been developed for English corpora.…”