Toward Suicidal Ideation Detection with Lexical Network Features and Machine Learning

Abstract

In this study, we introduce a new network feature for detecting suicidal ideation from clinical texts and conduct various additional experiments to enrich the state of knowledge. We evaluate statistical features with and without stopwords, use lexical networks for feature extraction and classification, and compare the results with standard machine learning methods using a logistic classifier, a neural network, and a deep learning method. We utilize three text collections. The first two contain transcriptions of interviews conducted by experts with suicidal (n=161 patients that experienced severe ideation) and control subjects (n=153). The third collection consists of interviews conducted by experts with epilepsy patients, with a few of them admitting to experiencing suicidal ideation in the past (32 suicidal and 77 control). The selected methods detect suicidal ideation with an average area under the curve (AUC) score of 95% on the merged collection with high suicidal ideation, and the trained models generalize over the third collection with an average AUC score of 69%. Results reveal that lexical networks are promising for classification and feature extraction as successful as the deep learning model. We also observe that a logistic classifier’s performance was comparable with the deep learning method while promising explainability.

Cognitive networks detect structural patterns and emotional complexity in suicide notes

“…It represents an extreme behavior that often arises from negative perceptions and distorted mindsets about the self, others and life itself (Rizk et al, 2019 ; Oquendo et al, 2020 ). Despite recent progress in using Big Data and natural language processing for the automatic detection and discrimination of suicide notes from other types of digital texts (Schoene and Dethlefs, 2016 ; Bayram et al, 2022 ), more work is required to understand how people who completed suicide organize and express emotions in their final writings (Palmier-Claus et al, 2012 ; Hallensleben et al, 2019 ). This challenging task requires the adoption of interpretable algorithms (as opposed to “black box" models such as deep neutral networks) that afford a greater understanding of the mental patterns and ways of thinking expressed in the final notes of those who completed suicide.…”