In Search of State and Trait Emotion Markers in Mobile-Sensed Language: Field Study

Abstract

Background Emotions and mood are important for overall well-being. Therefore, the search for continuous, effortless emotion prediction methods is an important field of study. Mobile sensing provides a promising tool and can capture one of the most telling signs of emotion: language. Objective The aim of this study is to examine the separate and combined predictive value of mobile-sensed language data sources for detecting both momentary emotional experience as well as global individual differences in emotional traits and depression. Methods In a 2-week experience sampling method study, we collected self-reported emotion ratings and voice recordings 10 times a day, continuous keyboard activity, and trait depression severity. We correlated state and trait emotions and depression and language, distinguishing between speech content (spoken words), speech form (voice acoustics), writing content (written words), and writing form (typing dynamics). We also investigated how well these features predicted state and trait emotions using cross-validation to select features and a hold-out set for validation. Results Overall, the reported emotions and mobile-sensed language demonstrated weak correlations. The most significant correlations were found between speech content and state emotions and between speech form and state emotions, ranging up to 0.25. Speech content provided the best predictions for state emotions. None of the trait emotion–language correlations remained significant after correction. Among the emotions studied, valence and happiness displayed the most significant correlations and the highest predictive performance. Conclusions Although using mobile-sensed language as an emotion marker shows some promise, correlations and predictive R2 values are low.

Ethical Considerations and Checklist for Affective Research With Wearables

“…We believe that participants should be compensated for the time devoted to the study. Paying participants with specific compensation structures corresponding to the level of involvement in the study is a well-known strategy in research using Experience Sampling Methods [54], [55].…”

Emotional Granularity is Associated with Daily Experiential Diversity

“…In Study 2 (Hoemann, Fan, et al, 2020), English-speaking participants rated the intensity of their experience at each prompt, but only wrote about a small subset. In Study 3 (Carlier et al, 2021), Dutch-speaking participants rated the intensity of their experience at each prompt and could optionally record a verbal description. To quantify results across studies, we performed both meta-and integrative data analyses of our findings.…”

“…Study 3 was approved by the KU Leuven Social and Societal Ethics Committee (protocol G-2018-01-1095) and has been reported in detail in Carlier et al (2021).…”

Using freely generated labels instead of rating scales to assess emotion in everyday life

“…Importantly, however, none of these studies of natural language used descriptions of current experience elicited from participants, leaving open the possibility that direct and intentional reports will index participant-reported affect from moment to moment. Recent work examining language use when participants are asked to describe what they had recently done and how it made them feel showed equivocal findings in this respect (Carlier et al, 2021). Yet examining this question with shorter-format descriptions (i.e., individual labels) may provide a more fruitful approach to validating language-based methods, and to potentially deploying them across a larger range of (clinical) contexts.…”