Identifying Moments of Change from Longitudinal User Text

Abstract

Identifying changes in individuals' behaviour and mood, as observed via content shared on online platforms, is increasingly gaining importance. Most research to-date on this topic focuses on either: (a) identifying individuals at risk or with a certain mental health condition given a batch of posts or (b) providing equivalent labels at the post level. A disadvantage of such work is the lack of a strong temporal component and the inability to make longitudinal assessments following an individual's trajectory and allowing timely interventions. Here we define a new task, that of identifying moments of change in individuals on the basis of their shared content online. The changes we consider are sudden shifts in mood (switches) or gradual mood progression (escalations). We have created detailed guidelines for capturing moments of change and a corpus of 500 manually annotated user timelines (18.7K posts). We have developed a variety of baseline models drawing inspiration from related tasks and show that the best performance is obtained through context aware sequential modelling. We also introduce new metrics for capturing rare events in temporal windows.

Predicting Moments of Mood Changes Overtime from Imbalanced Social Media Data

“…As per (Tsakalidis et al, 2022b), the evaluation is carried out using two types of metrics. The first one is post-level metrics, which assesses the model's performance using precision, recall, and F1 score.…”

“…In this paper, we explain our approach to the CLPsych (Tsakalidis et al, 2022a) shared task, which consists of two subtasks, as follows: Subtask A: Subtask A tries to capture those moments when a user's mood deviates from their baseline mood based on a user's postings throughout a specific time period -this is a post-level sequential classification task. The full task description can be found in (Tsakalidis et al, 2022b). Subtask B: A user-level classification task on predicting the degree of suicide risk.…”

Exploring transformers and time lag features for predicting changes in mood over time

“…They found that adding the year as a prefix to the input aided learning of seen facts, improving performance on predictions of future events. Tsakalidis et al (2022b) identify individuals' changes in mental health over time. This temporal dimension can be helpful in monitoring clinical outcomes and it can also help online platform moderators prioritize interventions depending on an individual's vulnerability at a certain moment in time.…”

“…We incorporate such information both through sequence models such as LSTMs (Hochreiter and Schmidhuber, 1997) that encode and preserve information from previous data points to make predictions, as well as through explicit custom features representing the time between data points, which we refer to as time lag features. We choose RoBERTa as a base for our models, as (Tsakalidis et al, 2022b) find BERT-based models perform well on this task, and RoBERTa models frequently outperform BERT in practice (Liu et al, 2019).…”

“…With the ubiquity of data online come opportunities for studying and providing support to individuals and communities. For example, a user's posts on Reddit fora may reveal information about that user's emotional state over time (Tsakalidis et al, 2022b). Additionally, these tasks may seek to make early predictions about mental states, allowing for prompt intervention when needed (Losada et al, 2020).…”