Prediction Models for Suicide Attempts and Deaths

Abstract

IMPORTANCESuicide prediction models have the potential to improve the identification of patients at heightened suicide risk by using predictive algorithms on large-scale data sources. Suicide prediction models are being developed for use across enterprise-level health care systems including the US Department of Defense, US Department of Veterans Affairs, and Kaiser Permanente.OBJECTIVES To evaluate the diagnostic accuracy of suicide prediction models in predicting suicide and suicide attempts and to simulate the effects of implementing suicide prediction models using population-level estimates of suicide rates.EVIDENCE REVIEW A systematic literature search was conducted in MEDLINE, PsycINFO, Embase, and the Cochrane Library to identify research evaluating the predictive accuracy of suicide prediction models in identifying patients at high risk for a suicide attempt or death by suicide. Each database was searched from inception to August 21, 2018. The search strategy included search terms for suicidal behavior, risk prediction, and predictive modeling. Reference lists of included studies were also screened. Two reviewers independently screened and evaluated eligible studies.FINDINGS From a total of 7306 abstracts reviewed, 17 cohort studies met the inclusion criteria, representing 64 unique prediction models across 5 countries with more than 14 million participants. The research quality of the included studies was generally high. Global classification accuracy was good (Ն0.80 in most models), while the predictive validity associated with a positive result for suicide mortality was extremely low (Յ0.01 in most models). Simulations of the results suggest very low positive predictive values across a variety of population assessment characteristics.CONCLUSIONS AND RELEVANCE To date, suicide prediction models produce accurate overall classification models, but their accuracy of predicting a future event is near 0. Several critical concerns remain unaddressed, precluding their readiness for clinical applications across health systems.

Reaching Those at Highest Risk for Suicide: Development of a Model Using Machine Learning Methods for use With Native American Communities

“…This may limit the applicability of these flags to ongoing data collection as these time trends may indicate differences in feature importance at different chronologic time. In most suicide risk identification projects due to suicide’s low base rate, our cutoff score based on the ridge yielded positive predictive value of only 0.12, which is consistent with other models in the literature (Belsher et al, ), and indicates a need for further exploration of the algorithms validity and whether this is an acceptable level of PPV within the CL system. We were not able to compare the performance of the algorithm to other forms of clinical risk assessment, something that could be addressed in a future study.…”

“…Ultimately, implementation of this algorithm will be guided by the local Apache CL team and focused on studying this algorithm within a clinical framework, following recent recommendations (Belsher et al, ). An algorithm that indicates risk 24 months out in time may not be clinically useful.…”

“…A challenge in suicide research is that PPV and NPV are both impacted by the underlying prevalence of the health issue. Thus with rare events, such as suicide attempt and death, PPV and NPV are often quite low (Belsher et al, ). Following recent work by Kessler et al (), Walsh et al (), and Simon et al (), we also looked at risk concentration plots to examine whether cases had higher predicted probabilities than controls.…”

Predicting 3‐month risk for adolescent suicide attempts among pediatric emergency department patients

“…The prediction model AUCs for the full sample, the sample of adolescents who did not report recent SI at baseline, and the subsamples of males and females each ranged between 0.84 and 0.89, which can be considered excellent classification accuracy (Hosmer et al., ), and contrasts with the disappointing performance of previous single risk factor approaches to suicide risk prediction (Franklin et al., ). Although the heterogeneity of suicide risk factors and the low base rates of SAs and suicide are challenges to risk stratification (Belsher et al., ), findings suggest that a multivariable prediction model can be useful for the short‐term prediction of adolescent SAs. However, of equal or greater importance, these models identify potentially important targets for clinical risk evaluation and prevention.…”

Cytomegalovirus Seropositivity and Suicidal Behavior: A Mini-Review

“…Although suicide research has made many advances [1], suicide remains an unpredictable [2] but leading cause of death. There are many known indicators/risk factors that can alert physicians' of those at risk of suicide, including clinical (e.g., psychiatric illness, substance use, previous attempts, medical illness), demographic (e.g., male sex, older age, living alone), genetic, and psychological (e.g., unemployment, interpersonal conflict) factors.…”