The kernel of truth in text-based personality assessment: A meta-analysis of the relations between the Big Five and the Linguistic Inquiry and Word Count (LIWC).

Abstract

The Linguistic Inquiry and Word Count (LIWC) is a popular closed-vocabulary text analysis software program that is used to understand whether individuals' use of linguistic categories (i.e., word categories, such as negative affect) depends on their personality traits. Here, we present the first meta-analysis of the relations between the Big Five personality traits and 52 linguistic categories of the English language. Across 31 eligible samples (n = 85,724), the results showed that (a) self-reported personality traits are significantly correlated with linguistic categories, but the effect sizes are relatively small (the strongest effect sizes between the Big Five and linguistic categories ranged from |ρ| = .08 to .14, and the 52 LIWC categories explained on average 5.1% of personality variance); (b) observerreported personality traits are significantly correlated with linguistic categories, with the effect sizes being small-to-medium (|ρ| = .18-.39, explaining 38.5% of personality variance); (c) 20 linguistic categories (out of 260; 5 Personality Traits × 52 LIWC Categories) correlated both with self-and observer-reported personality traits (the "kernel of truth" in linguistic markers of personality); and (d) 10 study, sample, and task characteristics significantly moderated the correlations of the linguistic categories with personality traits, showing that the effect sizes were mainly stronger for longer texts and older LIWC versions, among others. Public Significance StatementThis meta-analysis identifies the linguistic categories (i.e., word categories, such as negative affect) that individuals use depending on their personality traits, as well as the linguistic categories that other people use to draw personality inferences. Individuals indeed use specific linguistic categories depending on their personality traits and others use specific linguistic categories to draw personality inferences, but those relations are dependent on study and tasks characteristics (e.g., text length, Linguistic Inquiry and Word Count version).

Verbal behavior and the future of social science.

“…Additionally, the growing toolkit of NLP methods will continue to provide unique methods that serve particular research goals to varying degrees. For example, relatively simple word counting methods are generally more explainable, transparent, and scalable than more computationally intensive "black box" models like neural networks, providing great value to researchers interested in explanation over prediction of psychological phenomena (Koutsoumpis et al, 2022). However, such methods are generally noisier and provide less value for scholars who require greater accuracy to achieve more predictive or complex psychometric goals (Kjell et al, 2022;Yarkoni & Westfall, 2017).…”

An integrative conceptual review of multiperspective frameworks in personality research and a roadmap for extended applications in organizational psychology.

“…First, more attention is needed to establish the validity of these novel approaches for operationalizing personality. Although these efforts have begun (e.g., Eichstaedt et al, 2021; Koutsoumpis et al, 2022; Park et al, 2015), opportunities abound to discern the appropriateness and efficacy of AI and other computing algorithms as valid raters of personality. Several AI techniques show initial evidence of convergence with self-reported measures, despite concerns that early forms of this technology cannot discriminate adequately between traits (Phan & Rauthmann, 2021).…”

A systematic evaluation of text mining methods for short texts: Mapping individuals’ internal states from online posts

“…Past research has demonstrated that this assumption holds in many cases, with speech and writing reflecting personality traits, sociodemographic characteristics, personal values, and moral concerns (Boyd et al, 2015 ; Matsuo et al, 2019 ; Schultheiss, 2013 ; Schwartz et al, 2013 ; Tausczik & Pennebaker, 2010 ). Textual measures have mostly been validated against self-reports, showing different levels of correlation on a range of internal states (Boyd et al, 2015 ; Kennedy et al, 2021 ; Koutsoumpis et al, 2022 ; Lykousas et al, 2019 ; Malko et al, 2021 ; Matsuo et al, 2019 ; Mozes et al, 2021 ; Pellert et al, 2022 ).…”

“…This assumption underlies the gold standard of quantitative text analysis in the social sciences (Iliev et al, 2015 ; Nelson et al, 2021 ; Tausczik & Pennebaker, 2010 ), which involves defining theoretical concepts of interest (Kennedy et al, 2022 ) prior to the selection of a textual corpus which will be analyzed with the help of multiple trained human coders (Krippendorff, 2004a ). When the goal is inference on internal states, the assumption becomes more specific: external observers can use cues in verbal behavior to deduce individuals’ internal states (Koutsoumpis et al, 2022 ). On the one hand, understanding how certain states are reflected in language is a distinctly human ability (Iliev et al, 2015 ; Kennedy et al, 2022 ).…”

“…On the other hand, psychological research has noted an asymmetry between how individuals perceive their own states and how external observers assess these very states (Vazire, 2010 ). This asymmetry can result in a higher correlation between textual features and observer reports than between textual features and self-perceptions of one’s traits and states (Koutsoumpis et al, 2022 ). Furthermore, certain internal states appear to be reflected more clearly in text than others (Kennedy et al, 2021 ).…”