Monitoring the Twitter sentiment during the Bulgarian elections

What are the limits of automated Twitter sentiment classification? We analyze a large set of manually labeled tweets in different languages, use them as training data, and construct automated classification models. It turns out that the quality of classification models depends much more on the quality and size of training data than on the type of the model trained. Experimental results indicate that there is no statistically significant difference between the performance of the top classification models. We quantify the quality of training data by applying various annotator agreement measures, and identify the weakest points of different datasets. We show that the model performance approaches the inter-annotator agreement when the size of the training set is sufficiently large. However, it is crucial to regularly monitor the self- and inter-annotator agreements since this improves the training datasets and consequently the model performance. Finally, we show that there is strong evidence that humans perceive the sentiment classes (negative, neutral, and positive) as ordered.

show abstract

“…However, tweets which are "too close" to the hyperplane are considered neutral. Various realizations of "too close" are described in [40,41].…”

Section: Classification Models Performancementioning

confidence: 99%

Multilingual Twitter Sentiment Classification: The Role of Human Annotators

Mozetič¹,

Grčar²,

Smailović³

2016

PLoS ONE

153

133

View full text Add to dashboard Cite

show abstract

“…The raw tweet data is based on collected 16,077 tweets posted between 29 April 2013 to 27 May 2013 in total [27]. Out of this sample, we extracted 5,817 tweets from 29 April to 11 May 2013.…”

Section: Data Descriptionmentioning

confidence: 99%

Twitter-Based Analysis of the Dynamics of Collective Attention to Political Parties

et al. 2015

View full text Add to dashboard Cite

Large-scale data from social media have a significant potential to describe complex phenomena in the real world and to anticipate collective behaviors such as information spreading and social trends. One specific case of study is represented by the collective attention to the action of political parties. Not surprisingly, researchers and stakeholders tried to correlate parties' presence on social media with their performances in elections. Despite the many efforts, results are still inconclusive since this kind of data is often very noisy and significant signals could be covered by (largely unknown) statistical fluctuations. In this paper we consider the number of tweets (tweet volume) of a party as a proxy of collective attention to the party, identify the dynamics of the volume, and show that this quantity has some information on the election outcome. We find that the distribution of the tweet volume for each party follows a log-normal distribution with a positive autocorrelation of the volume over short terms, which indicates the volume has large fluctuations of the log-normal distribution yet with a short-term tendency. Furthermore, by measuring the ratio of two consecutive daily tweet volumes, we find that the evolution of the daily volume of a party can be described by means of a geometric Brownian motion (i.e., the logarithm of the volume moves randomly with a trend). Finally, we determine the optimal period of averaging tweet volume for reducing fluctuations and extracting short-term tendencies. We conclude that the tweet volume is a good indicator of parties' success in the elections when considered over an optimal time window. Our study identifies the statistical nature of collective attention to political issues and sheds light on how to model the dynamics of collective attention in social media.

show abstract

“…We have already applied the same sentiment classification methodology in various domains, such as: (i) to study the emotional dynamics of Facebook comments on conspiracy theories (in Italian) [38], (ii) to compare the sentiment leaning of different network communities towards various environmental topics [39], and (iii) to monitor the sentiment about political parties before and after the elections (in Bulgarian) [40].…”

Section: Introductionmentioning

confidence: 99%

The Effects of Twitter Sentiment on Stock Price Returns

Ranco

Aleksovski

Caldarelli³

et al. 2015

PLoS ONE

279

188

View full text Add to dashboard Cite

Social media are increasingly reflecting and influencing behavior of other complex systems. In this paper we investigate the relations between a well-known micro-blogging platform Twitter and financial markets. In particular, we consider, in a period of 15 months, the Twitter volume and sentiment about the 30 stock companies that form the Dow Jones Industrial Average (DJIA) index. We find a relatively low Pearson correlation and Granger causality between the corresponding time series over the entire time period. However, we find a significant dependence between the Twitter sentiment and abnormal returns during the peaks of Twitter volume. This is valid not only for the expected Twitter volume peaks (e.g., quarterly announcements), but also for peaks corresponding to less obvious events. We formalize the procedure by adapting the well-known “event study” from economics and finance to the analysis of Twitter data. The procedure allows to automatically identify events as Twitter volume peaks, to compute the prevailing sentiment (positive or negative) expressed in tweets at these peaks, and finally to apply the “event study” methodology to relate them to stock returns. We show that sentiment polarity of Twitter peaks implies the direction of cumulative abnormal returns. The amount of cumulative abnormal returns is relatively low (about 1–2%), but the dependence is statistically significant for several days after the events.

show abstract

Monitoring the Twitter sentiment during the Bulgarian elections

Cited by 33 publications

References 24 publications

Multilingual Twitter Sentiment Classification: The Role of Human Annotators

Multilingual Twitter Sentiment Classification: The Role of Human Annotators

Twitter-Based Analysis of the Dynamics of Collective Attention to Political Parties

The Effects of Twitter Sentiment on Stock Price Returns

Contact Info

Product

Resources

About