Combining Psychological Theory with Language Models for Suicide Risk Detection

Abstract

Recent years saw a dramatic increase in the popularity of online counseling services providing emergency mental health support. This paper provides a new language model for automatic detection of suicide risk in online chat sessions between help-seekers and counselors. The model adapts a hierarchical BERT language model for this task. It extends the state of the art in capturing aspects of the conversation structure in the counseling session and in integrating psychological theory into the model. We test the performance of our approach in a leading national online counseling service that operates in the Hebrew language. Our model outperformed other non-hierarchical approaches from the literature, achieving a 0.76 F2 score and 0.92 ROC-AUC. Moreover, we demonstrate our model's superiority over strong baselines even early on in the conversation, which is key for real-time detection in the field. This is a first step towards incorporating suicide predictive models in online support services and advancing NLP tools for resource-bounded languages.

Predicting suicide risk in real‐time crisis hotline chats integrating machine learning with psychological factors: Exploring the black box

“…We randomly split the labeled Sahar dataset into three sets: train (70%), validate (15%), and test (15%). These datasets were used throughout experiments described in detail in Bialer et al (2022) and Izmaylov et al (2023). To evaluate the model performance, we used ROC-AUC, widely employed in suicide detection research (Bernert et al, 2020).…”

“…We incorporated the SRF lexicon as a pretraining task. As described in a previous study (Izmaylov et al, 2023), we first chose a 5-dimension representation that outperformed the 20-dimension representation on the validation set, leading us to use this representation in the subsequent pre-training phase. In the second step, the self-supervised knowledge task (SSK task) was applied as a new pretraining task for predicting Sahar conversations in the SRF representation space.…”