Evaluation of Conversational Skills for Commonsense

Baymurzina, Dilyara; Kuratov, Yurii; Kuznetsov, D.; Kornev, Daniel; Burtsev, Mikhail

doi:10.28995/2075-7182-2021-20-1002-1011

Cited by 2 publications

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Weighted F 1 is reported because it was used as the main quality measure in the original paper. Existing weighted F 1 SOTA was achieved by shallow-and-wide CNN with ELMo embeddings [145]. Existing macro F 1 SOTA was achieved by fine-tuned RuBERT [65].…”

Section: Classification Modelmentioning

confidence: 99%

Pulse of the Nation: Observable Subjective Well-Being in Russia Inferred from Social Network Odnoklassniki

Smetanin

2022

Mathematics

View full text Add to dashboard Cite

Policymakers and researchers worldwide are interested in measuring the subjective well-being (SWB) of populations. In recent years, new approaches to measuring SWB have begun to appear, using digital traces as the main source of information, and show potential to overcome the shortcomings of traditional survey-based methods. In this paper, we propose the formal model for calculation of observable subjective well-being (OSWB) indicator based on posts from a social network, which utilizes demographic information and post-stratification techniques to make the data sample representative by selected characteristics of the general population. We applied the model on the data from Odnoklassniki, one of the largest social networks in Russia, and obtained an OSWB indicator representative of the population of Russia by age and gender. For sentiment analysis, we fine-tuned several language models on RuSentiment and achieved state-of-the-art results. The calculated OSWB indicator demonstrated moderate to strong Pearson’s (r=0.733, p=0.007, n=12) correlation and strong Spearman’s (rs=0.825, p=0.001, n=12) correlation with a traditional survey-based Happiness Index reported by Russia Public Opinion Research Center, confirming the validity of the proposed approach. Additionally, we explored circadian (24 h) and circaseptan (7 day) patterns, and report several interesting findings for the population of Russia. Firstly, daily variations were clearly observed: the morning had the lowest level of happiness, and the late evening had the highest. Secondly, weekly patterns were clearly observed as well, with weekends being happier than weekdays. The lowest level of happiness occurs in the first three weekdays, and starting on Thursday, it rises and peaks during the weekend. Lastly, demographic groups showed different levels of happiness on a daily, weekly, and monthly basis, which confirms the importance of post-stratification by age group and gender in OSWB studies based on digital traces.

show abstract

Section: Classification Modelmentioning

confidence: 99%