Characterisation of COVID-19-Related Tweets in the Croatian Language: Framework Based on the Cro-CoV-cseBERT Model

Abstract

This study aims to provide insights into the COVID-19-related communication on Twitter in the Republic of Croatia. For that purpose, we developed an NL-based framework that enables automatic analysis of a large dataset of tweets in the Croatian language. We collected and analysed 206,196 tweets related to COVID-19 and constructed a dataset of 10,000 tweets which we manually annotated with a sentiment label. We trained the Cro-CoV-cseBERT language model for the representation and clustering of tweets. Additionally, we compared the performance of four machine learning algorithms on the task of sentiment classification. After identifying the best performing setup of NLP methods, we applied the proposed framework in the task of characterisation of COVID-19 tweets in Croatia. More precisely, we performed sentiment analysis and tracked the sentiment over time. Furthermore, we detected how tweets are grouped into clusters with similar themes across three pandemic waves. Additionally, we characterised the tweets by analysing the distribution of sentiment polarity (in each thematic cluster and over time) and the number of retweets (in each thematic cluster and sentiment class). These results could be useful for additional research and interpretation in the domains of sociology, psychology or other sciences, as well as for the authorities, who could use them to address crisis communication problems.

Retweet Prediction based on Heterogeneous Data Sources: The Combination of Text and Multilayer Network Features

“…The seminal work of [41] contributed to the emergence of numerous variants of text representation models in terms of low-dimensional vectors in continuous space-embeddings, where embeddings allow semantically related linguistic units to be represented with similar vector representations. As described in [8] the first generation was characterised by shallow language models, such as Word2Vec [41], Doc2Vec [42], GloVe [43] and fastText [44]. They have some shortcomings, such as static embeddings in which multiple concepts (i.e., different meanings of the same entity, polysemy) are not represented by different embedding vectors, or poor performance in new domains.…”

“…For example, the outbreak of the COVID-19 disease caused a significant increase in social media usage among the public and it seriously affected the public's understanding of the COVID-19 risk [7]. In some countries there were many negative attitudes toward vaccines and anti-pandemic measures promoted on social networks [8]. Therefore, information spreading analysis during the global crisis is of great importance as one step of social media monitoring (infoveillance).…”

“…Consequently, it is one of the most studied social networks. Recently, especially for the monitoring and tracking different aspects of healthcare information and public disease [8,10,11]. Among all the user behavior in social media, the retweet is considered one of the primary functions for spreading information on Twitter [12,13].…”

“…The text features are represented as a low dimensional vector (embedding) that captures its semantic and structure. More specifically, we adopt a BERT-based language model, namely Cro-CoV-cseBERT [8] for tweets' representation as embeddings, which we use as the set of text features.…”

Infoveillance of the Croatian Online Media During the COVID-19 Pandemic: One-Year Longitudinal Study Using Natural Language Processing

“…Most studies employed different NLP techniques for capturing specific aspects of the COVID-19 content published online. For discovering public perceptions, opinions, and attitudes toward specific COVID-19-related topics, researchers commonly combine topic modeling and sentiment analysis [21,[26][27][28], which are also occasionally combined with named entity recognition (NER) [29].…”