Searching For Prosociality in Qualitative Data: Comparing Manual, Closed-Vocabulary, and Open-Vocabulary Methods

Abstract

Though most people present themselves as possessing prosocial traits, people differ in the extent to which they actually act prosocially in everyday life. Qualitative data that was not ostensibly collected to measure prosociality might contain information about prosocial dispositions that is not distorted by self-presentation concerns. This paper seeks to characterise charitable donors from qualitative data. We compared a manual approach of extracting predictors from participants’ self-described personal strivings to two automated approaches: A summation of words predefined as prosocial and a support vector machine classifier. Although variables extracted by the support vector machine predicted donation behaviour well in the training sample (N = 984), virtually no variables from any method significantly predicted donations in a holdout sample (N = 496). Raters’ attempts to predict donations to charity based on reading participants’ personal strivings were also unsuccessful. However, raters’ predictions were associated with past charitable involvement. In sum, predictors derived from personal strivings did not robustly explain variation in charitable behaviour, but personal strivings may nevertheless contain some information about trait prosociality. The sparseness of personal strivings data, rather than the irrelevance of open-ended text or individual differences in goal pursuit, likely explains their limited value in predicting prosocial behaviour.