Reply to Garcia et al.: Common mistakes in measuring frequency-dependent word characteristics

Dodds, Peter Sheridan; Clark, Eric M.; Desu, Suma; Frank, Morgan R.; Reagan, Andrew J.; Williams, Jake Ryland; Mitchell, Lewis; Harris, Kameron Decker; Kloumann, Isabel M.; Bagrow, James P.; Megerdoomian, Karine; McMahon, Matthew T.; Tivnan, Brian F.; Danforth, Christopher M.

doi:10.1073/pnas.1505647112

Cited by 8 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the problems with the "neutral" words in the LIWC set are immediate: these are not emotionally neutral words. The range of scores in LabMT for these 0score words in LIWC formed the basis for Garcia et al's response to [5], and we point out here that the authors must have not looked at the words, and all-too-common problem in studies using sentiment analysis [16,17].…”

Section: Resultsmentioning

confidence: 95%

“…By contrast, LabMT scores represent 50 ratings of each word. For an in depth comparison, see reference [17].…”

Section: Resultsmentioning

confidence: 99%

“…The axes in the bar plots are not of the same height, due to large mismatches in the number of words in the dictionaries, and we note the maximum height of the bar in the upper left of such plots. Detailed analysis of Panel C can be found in [17]. We provide a table for each off-diagonal panel in the S2 Appendix with the words whose scores exhibit the greatest mismatch, and a subset of these tables in While it would be simple to correct these ratings in the MPQA dictionary going forward, we have are naturally led to be concerned about existing work using MPQA.…”

Section: Resultsmentioning

confidence: 99%

See 2 more Smart Citations

Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs

et al. 2017

Self Cite

View full text Add to dashboard Cite

The emergence and global adoption of social media has rendered possible the real-time estimation of population-scale sentiment, an extraordinary capacity which has profound implications for our understanding of human behavior. Given the growing assortment of sentiment-measuring instruments, it is imperative to understand which aspects of sentiment dictionaries contribute to both their classification accuracy and their ability to provide richer understanding of texts. Here, we perform detailed, quantitative tests and qualitative assessments of 6 dictionary-based methods applied to 4 different corpora, and briefly examine a further 20 methods. We show that while inappropriate for sentences, dictionary-based methods are generally robust in their classification accuracy for longer texts. Most importantly they can aid understanding of texts with reliable and meaningful word shift graphs if (1) the dictionary covers a sufficiently large portion of a given text's lexicon when weighted by word usage frequency; and (2) words are scored on a continuous scale.

show abstract

Section: Resultsmentioning

confidence: 95%

“…By contrast, LabMT scores represent 50 ratings of each word. For an in depth comparison, see reference [17].…”

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, if the word "joy" is noted five times in an email, it would be counted as one type and five tokens. Although there is general agreement that LPB is observable in multiple languages, there has been an ongoing debate about whether LPB happens on the level of types or on the level of tokens (10,11,(15)(16)(17). In this paper we study the phenomenon on the level of tokens only.…”

Section: Significancementioning

confidence: 99%

Linguistic positivity in historical texts reflects dynamic environmental and psychological factors

Iliev

Hoover

Dehghani

et al. 2016

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

People use more positive words than negative words. Referred to as "linguistic positivity bias" (LPB), this effect has been found across cultures and languages, prompting the conclusion that it is a panhuman tendency. However, although multiple competing explanations of LPB have been proposed, there is still no consensus on what mechanism(s) generate LPB or even on whether it is driven primarily by universal cognitive features or by environmental factors. In this work we propose that LPB has remained unresolved because previous research has neglected an essential dimension of language: time. In four studies conducted with two independent, time-stamped text corpora (Google books Ngrams and the New York Times), we found that LPB in American English has decreased during the last two centuries. We also observed dynamic fluctuations in LPB that were predicted by changes in objective environment, i.e., war and economic hardships, and by changes in national subjective happiness. In addition to providing evidence that LPB is a dynamic phenomenon, these results suggest that cognitive mechanisms alone cannot account for the observed dynamic fluctuations in LPB. At the least, LPB likely arises from multiple interacting mechanisms involving subjective, objective, and societal factors. In addition to having theoretical significance, our results demonstrate the value of newly available data sources in addressing long-standing scientific questions.Pollyanna hypothesis | positivity in language | automated text analysis | subjective happiness F or a naive observer, human language use often appears to be a function of utility and context. We craft our enunciations according to our intentions and, aside from correlations between language and environmental and experiential cues, we seem to be masters of our words. However, as social scientists have long recognized, linguistic patterns are also a function of social, cultural, and cognitive factors (1-5).One of the more striking examples of such linkages comes from research on the affective content of language revealing that people use positive words more frequently than negative words, an effect first reported by Zajonc in 1968 (6) and subsequently replicated many times (7-11). This phenomenon, initially referred to as the "Pollyanna hypothesis" (7) and more recently as "linguistic positivity bias" (LPB) (9), has been demonstrated across numerous languages and corpora (7,11,12), leading to the conclusion that it is a panhuman tendency.However, although the existence of LPB is well-established, there is surprisingly little consensus on what mechanisms are responsible for the effect. Among the mechanisms proposed are universal cognitive biases (6, 7), affective states (12), objective circumstances (8), and societal norms (9, 10, 12). However, research has been unable to establish which of these mechanisms drives LPB and whether the effect might be driven by interaction between a subset of these or other factors. One reason for this uncertainty is that previous investigations of LPB ha...

show abstract

“…Some examples include comparing the happiness of users to their online social networks 11,12 , identifying detailed predictors of mood through social media feeds 7 , predicting cognitive distortions expressed among groups at-risk of mental health disorders 13 , tracking the emotions of social media users at high resolution 14,15 , and mapping negative affectivity among users with internalizing disorders 16 . Collectively, these studies demonstrate the feasibility and value of using sentiment analysis on social media data to study societal mood and well-being, as well as biomedical signals among social media users that can provide useful proxies for mental health 13,[17][18][19] . In fact, these approaches may be especially useful considering the speed with which the pandemic became an acute socio-economic phenomenon, the pervasiveness of COVID-19 related content available online, and the natural reaction of many to post on social media about pandemic-related events.…”

Section: Introductionmentioning

confidence: 99%

Social Media Insights Into US Mental Health Amid the COVID-19 Pandemic. A Longitudinal Twitter Analysis (JANUARY-APRIL 2020)

Valdez

Thij

Bathina

et al. 2020

Preprint

View full text Add to dashboard Cite

BackgroundThe COVID-19 pandemic led to unprecedented mitigation efforts that disrupted the daily lives of millions. Beyond the general health repercussions of the pandemic itself, these measures also present a significant challenge to the world’s mental health and healthcare systems. Considering traditional survey methods are time-consuming and expensive, we need timely and proactive data sources to respond to the rapidly evolving effects of health policy on our population’s mental health. Significant pluralities of the US population now use social media platforms, such as Twitter, to express the most minute details of their daily lives and social relations. This behavior is expected to increase during the COVID-19 pandemic, rendering social media data a rich field from which to understand personal wellbeing.PurposeBroadly, this study answers three research questions: RQ1: What themes emerge from a corpus of US tweets about COVID-19?; RQ2: To what extent does social media use increase during the onset of the COVID-19 pandemic?; and RQ3: Does sentiment change in response to the COVID-19 pandemic?MethodsWe analyzed 86,581,237 public domain English-language US tweets collected from an open-access public repository in three steps1. First, we characterized the evolution of hashtags over time using Latent Dirichlet Allocation (LDA) topic modeling. Second, we increased the granularity of this analysis by downloading Twitter timelines of a large cohort of individuals (n = 354,738) in 20 major US cities to assess changes in social media use. Finally, using this timeline data, we examined collective shifts in public mood in relation to evolving pandemic news cycles by analyzing the average daily sentiment of all timeline tweets with the Valence Aware Dictionary and sEntiment Reasoner (VADER) sentiment tool2.ResultsLDA topics generated in the early months of the dataset corresponded to major COVID-19 specific events. However, as state and municipal governments began issuing stay-at-home orders, latent themes shifted towards US-related lifestyle changes rather than global pandemic-related events. Social media volume also increased significantly, peaking during stay-at-home mandates. Finally, VADER sentiment analysis sentiment scores of user timelines were initially high and stable, but decreased significantly, and continuously, by late March.Discussion & ConclusionOur findings underscore the negative effects of the pandemic on overall population sentiment. Increased usage rates suggest that, for some, social media may be a coping mechanism to combat feelings of isolation related to long-term social distancing. However, in light of the documented negative effect of heavy social media usage on mental health, for many social media may further exacerbate negative feelings in the long-term. Thus, considering the overburdened US mental healthcare structure, these findings have important implications for ongoing mitigation efforts.

show abstract

Reply to Garcia et al.: Common mistakes in measuring frequency-dependent word characteristics

Cited by 8 publications

References 10 publications

Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs

Sentiment analysis methods for understanding large-scale texts: a case for using continuum-scored words and word shift graphs

Linguistic positivity in historical texts reflects dynamic environmental and psychological factors

Social Media Insights Into US Mental Health Amid the COVID-19 Pandemic. A Longitudinal Twitter Analysis (JANUARY-APRIL 2020)

Contact Info

Product

Resources

About