Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives.

Abstract

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer (n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry and Word Count [LIWC] and Psychiatric Content Analysis and Diagnosis [PCAD]). Although the computerized coding methods captured most of the emotion identified by raters (LIWC sensitivity = .88; PCAD sensitivity = .83), both over-identified emotional expression (LIWC positive predictive value = .31; PCAD positive predictive value = .19). Correlational analyses suggested better convergent and discriminant validity for LIWC. The results highlight previously unrecognized deficiencies in commonly used computerized content-analysis programs and suggest potential modifications to both programs that could improve overall accuracy of automated identification of emotional expression. Although the authors recognize these limitations, they conclude that LIWC is superior to PCAD for rapid identification of emotional expression in text. (PsycINFO Database Record (c) 2009 APA, all rights reserved).

“You’re as Sick as You Sound”: Using Computational Approaches for Modeling Speaker State to Gauge Illness and Recovery

“…So far, there is conflicting evidence that higher-level linguistic features are more helpful than shallow, lexical ones or speech signal information. In two studies, for instance, purely lexical features (as derived from lexical resources like LIWC) performed better than context-aware features, such as PCAD and the Gottschalk-Gleser scales [5,40]. On the other hand, Le Normand [28] shows that semantic and syntactic information derived from manually labeled speech acts can help target specific functions of prosody, which have been described previously as difficult for individuals with ASD to process.…”

“…The use of lexical resources to recognize expressions of emotion in text was also investigated in the work of Bantum and Owen [5]. They compare two automatic resources, LIWC and the Psychiatric Content Analysis and Diagnosis system (PCAD), based on the Gottschalk-Gleser scales mentioned above, for the recognition of positive and negative emotions, as well as more particular emotions of anxiety, anger, sadness, and optimism.…”

“…Besides the annotation costs, the validity of the annotation is important. In several studies, expert opinions are not always reliable [42,45], but neither are patients' self-reports [5].…”

“…Free spontaneous speech [28,42] 2. Free speech, albeit about a particular topic [5] 3. Proxy texts [45] 4.…”

Change-Point Analysis of the Public Mood in UK Twitter during the Brexit Referendum

“…The lexicon was designed to be applied to a wide range of different texts, including transcribed every day speech and email, making it suitable for application in social media domains. Furthermore, the lists were validated by independent judges, and found to have high levels (0.88 and 0.97 respectively) of sensitivity and specificity for all emotional expression words [16].…”