Predicting suicide risk in real‐time crisis hotline chats integrating machine learning with psychological factors: Exploring the black box

Abstract

BackgroundThis study addresses the suicide risk predicting challenge by exploring the predictive ability of machine learning (ML) models integrated with theory‐driven psychological risk factors in real‐time crisis hotline chats. More importantly, we aimed to understand the specific theory‐driven factors contributing to the ML prediction of suicide risk.MethodThe dataset consisted of 17,654 crisis hotline chat sessions classified dichotomously as suicidal or not. We created a suicide risk factors‐based lexicon (SRF), which encompasses language representations of key risk factors derived from the main suicide theories. The ML model (Suicide Risk‐Bert; SR‐BERT) was trained using natural language processing techniques incorporating the SRF lexicon.ResultsThe results showed that SR‐BERT outperformed the other models. Logistic regression analysis identified several theory‐driven risk factors significantly associated with suicide risk, the prominent ones were hopelessness, history of suicide, self‐harm, and thwarted belongingness.LimitationsThe lexicon is limited in its ability to fully encompass all theoretical concepts related to suicide risk, nor to all the language expressions of each concept. The classification of chats was determined by trained but non‐professionals in metal health.ConclusionThis study highlights the potential of how ML models combined with theory‐driven knowledge can improve suicide risk prediction. Our study underscores the importance of hopelessness and thwarted belongingness in suicide risk and thus their role in suicide prevention and intervention.

Applications of Large Language Models in the Field of Suicide Prevention: A Scoping Review (Preprint)

“…Of the 31 studies employing social media data, user or poster consent for the use of their data was sought in only one study [43]. Of the 12 studies (28%) that did not use social media data, 3 used data from crisis counselling apps or services [57][58][59], with the remainder a heterogenous collection of data from the National Violent Death Report System [50,60], educational or academic documents [51,61], or participant vignettes [49,52]. One study utilized research participant data gathered during the course of the study [62].…”

“…Additional uses included identifying precipitating events to suicide from death investigation narratives [60] and identifying self-harm from social media posts [77]. Prediction focused studies used LLMs utilizing Reddit [44,78,79], other social media [55], images posted on social media [43], crisis counselling data [58], or clinical data [78] predominantly to predict suicide risk. The remaining four model applications assessed support [48], education [51], and 'other' (i.e., the generation of mental health care plans & means restriction) [52,63].…”

“…To be considered as having meaningfully reported on ethical issues, studies needed to include discussion of the ethical considerations and their possible implications. Cursory mention of potential ethical issues without discussion of their implications was not considered sufficient (e.g., mention of LLMs as being a 'black box' without any further discussion of the implications [43,58] was not considered sufficient). Ethical considerations were discussed in 14 (33%) studies.…”

“…One study noted that these data biases or lack of diversity could lead to erroneous predictions that may exacerbate existing disparities in suicide prevention strategies [49]. Three studies noted lack of transparency, resulting from the opaque functioning of the underlying algorithms (termed 'black box') [49,58], with one noting a lack of governance in this regard [52]. Two studies raised concerns about LLMs threatening the client-clinician relationship, potentially leading to human-centered clinical interactions being replaced by digital alternatives.…”