Sentiment Analysis of Tweets using Unsupervised Learning Techniques and the K-Means Algorithm

Abstract

Today, web content such as images, text, speeches, and videos are user-generated, and social networks have become increasingly popular as a means for people to share their ideas and opinions. One of the most popular social media for expressing their feelings towards events that occur is Twitter. The main objective of this study is to classify and analyze the content of the affiliates of the Pension and Funds Administration (AFP) published on Twitter. This study incorporates machine learning techniques for data mining, cleaning, tokenization, exploratory analysis, classification, and sentiment analysis. To apply the study and examine the data, Twitter was used with the hashtag #afp, followed by descriptive and exploratory analysis, including metrics of the tweets. Finally, a content analysis was carried out, including word frequency calculation, lemmatization, and classification of words by sentiment, emotions, and word cloud. The study uses tweets published in the month of May 2022. Sentiment distribution was also performed in three polarity classes: positive, neutral, and negative, representing 22%, 4%, and 74% respectively. Supported by the unsupervised learning method and the K-Means algorithm, we were able to determine the number of clusters using the elbow method. Finally, the sentiment analysis and the clusters formed indicate that there is a very pronounced dispersion, the distances are not very similar, even though the data standardization work was carried out.

Clustering Algorithms in Sentiment Analysis Techniques in Social Media – A Rapid Literature Review

“…The SA algorithms are used to classify the polarity of a text as positive, negative, or neutral, based on the sentiment expressed in the text. Among their various contributions to SA, the specialized literature mentions:  Text classification [38], [31], [22]: SA algorithms are often used to classify text into different sentiment categories, such as positive, negative, or neutral. This is typically done using supervised learning algorithms, like Naive Bayes [2], [39], Support Vector Machine (SVM) [40], [41], Logistic Regression [41], [42] or DL algorithms like BERT, LSTM and CNN.…”

Text prediction recurrent neural networks using long short-term memory-dropout

“…The LSTM neural network is applicable to different areas of industry. For example, reseachers [14]- [17] proposed an approach for better economic forecasting and decision-making by using the k-means clustering algorithm to group banks with similar price trends, then, to train these grouped stocks, they used a neural network model of short-and long-term memory (LSTM) to perform static and dynamic prediction of stock prices and economic growth rates. Lin et al [18] developed a forecasting model with LSTM using decomposed data to obtain the prediction sequence to forecast the stock index price of the standard & poor's 500 indexes [19] and used the LSTM neural network model with the Adam algorithm to achieve better prediction results.…”

Search and classify topics in a corpus of text using the latent dirichlet allocation model

“…LDA is a probabilistic generative statistical model that assumes that each text is a distribution of topics and each topic is a distribution of words. As a result, given a text, the model seeks to determine the proportion of one variable given the value of another variable in order to maximize the parameters of the generative model [11], [12]. LDA is a topic modeling method widely used by academics and researchers in text classification.…”