The Veterans Health Administration REACH VET Program: Suicide Predictive Modeling in Practice

Matarazzo, Bridget B.; Eagan, Aaron; Landes, Sara J.; Mina, Liam K.; Clark, Kaily; Gerard, Georgia R.; McCarthy, John F.; Trafton, Jodie A.; Bahraini, Nazanin H.; Brenner, Lisa A.; Keen, Angela; Gamble, Stephanie A.; Lawson, Wendy; Katz, Ira R.; Reger, Mark A.

doi:10.1176/appi.ps.202100629

Cited by 10 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, it may be especially relevant when estimating patients’ risk of repeat suicidal behavior, for example after an ED visit for suicidal behavior, which could influence clinical decision-making at the point of care (eg, about discharge home or to outpatient care versus hospitalization). This rule may have less impact for other related prediction tasks, such as estimating patients’ risk of suicidal behavior after nonsuicide-related outpatient visits or broader population-based prediction efforts [ 25 ]. These results may also be less relevant for models that solely predict fatal self-harm or suicide deaths [ 26 , 27 ].…”

Section: Discussionmentioning

confidence: 99%

Determining Distinct Suicide Attempts From Recurrent Electronic Health Record Codes: Classification Study

Bentley,

Madsen,

Song

et al. 2024

JMIR Form Res

View full text Add to dashboard Cite

Background Prior suicide attempts are a relatively strong risk factor for future suicide attempts. There is growing interest in using longitudinal electronic health record (EHR) data to derive statistical risk prediction models for future suicide attempts and other suicidal behavior outcomes. However, model performance may be inflated by a largely unrecognized form of “data leakage” during model training: diagnostic codes for suicide attempt outcomes may refer to prior attempts that are also included in the model as predictors. Objective We aimed to develop an automated rule for determining when documented suicide attempt diagnostic codes identify distinct suicide attempt events. Methods From a large health care system’s EHR, we randomly sampled suicide attempt codes for 300 patients with at least one pair of suicide attempt codes documented at least one but no more than 90 days apart. Supervised chart reviewers assigned the clinical settings (ie, emergency department [ED] versus non-ED), methods of suicide attempt, and intercode interval (number of days). The probability (or positive predictive value) that the second suicide attempt code in a given pair of codes referred to a distinct suicide attempt event from its preceding suicide attempt code was calculated by clinical setting, method, and intercode interval. Results Of 1015 code pairs reviewed, 835 (82.3%) were nonindependent (ie, the 2 codes referred to the same suicide attempt event). When the second code in a pair was documented in a clinical setting other than the ED, it represented a distinct suicide attempt 3.3% of the time. The more time elapsed between codes, the more likely the second code in a pair referred to a distinct suicide attempt event from its preceding code. Code pairs in which the second suicide attempt code was assigned in an ED at least 5 days after its preceding suicide attempt code had a positive predictive value of 0.90. Conclusions EHR-based suicide risk prediction models that include International Classification of Diseases codes for prior suicide attempts as a predictor may be highly susceptible to bias due to data leakage in model training. We derived a simple rule to distinguish codes that reflect new, independent suicide attempts: suicide attempt codes documented in an ED setting at least 5 days after a preceding suicide attempt code can be confidently treated as new events in EHR-based suicide risk prediction models. This rule has the potential to minimize upward bias in model performance when prior suicide attempts are included as predictors in EHR-based suicide risk prediction models.

show abstract

Section: Discussionmentioning

confidence: 99%

Determining Distinct Suicide Attempts From Recurrent Electronic Health Record Codes: Classification Study

Bentley,

Madsen,

Song

et al. 2024

JMIR Form Res

View full text Add to dashboard Cite

show abstract

“…The reduction in suicide rates appears to be at least partly attributable to the implementation of several suicide prevention program initiatives from risk assessment to prevention interventions. For example, REACH VET (Recovery Engagement and Coordination for Veterans Enhanced Treatment) 27 launched in 2016 and uses predictive modeling to conduct suicide risk identification. In another example, RISK‐ID (Suicide Risk Identification), 28 launched in 2018; this program standardizes efforts to conduct suicide risk screening 29 …”

Section: Va Emergency Care Researchmentioning

confidence: 99%

Continuing the transformation: charting the path for the future delivery of Veteran emergency care

Ward

Kessler

Abel

et al. 2023

Academic Emergency Medicine

View full text Add to dashboard Cite

Important changes in the delivery of Veteran emergency care in the early 2000s in the Department of Veteran Affairs (VA) emergency departments and urgent care clinics substantially elevated the role of emergency medicine (EM) in Veteran health care. Focused on enhancing the quality of care, emergency care visits in both VA and non-VA (community) care locations have nearly doubled from the 1980s to more than 3 million visits in Fiscal Year 2022. Recognizing the need to plan for continued growth and the opportunity to address key research priorities, the VA Office of Emergency Medicine, together with the VA Health Services Research and Development Service, collaborated to convene a State of the Art Conference on Veteran Emergency Medicine (SAVE) in the winter of 2022. The goal of this conference was to identify research gaps and priorities for implementation of policies for three priority groups: geriatric Veterans, Veterans with mental health and substance use complaints, and Veterans presenting to non-VA (community) emergency care sites. In this article we discuss the rationale for the SAVE conference including a brief history of VA EM and the planning process and conclude with next steps for findings from the conference.

show abstract

“…While statistical risk prediction models have shown some promise in improving suicide risk prediction accuracy [ 55 – 58 ] and cost-effectiveness [ 59 ], their application in clinical settings remains limited. The U.S. Veterans’ Health Administration utilized a statistical model in the REACH VET program to identify at-risk veteran patients and link them with care [ 60 , 61 ]. Jaspr Health introduced a tablet-based digital assistant, featuring an AI-driven chatbot aiding ED patients with suicide risk and establishing evidence-based discharge plans, proving both feasible and acceptable [ 62 ].…”

Section: Introductionmentioning

confidence: 99%

Developing a clinical decision support system software prototype that assists in the management of patients with self-harm in the emergency department: protocol of the PERMANENS project

Mortier,

Amigo,

Bhargav

et al. 2024

BMC Psychiatry

View full text Add to dashboard Cite

Background Self-harm presents a significant public health challenge. Emergency departments (EDs) are crucial healthcare settings in managing self-harm, but clinician uncertainty in risk assessment may contribute to ineffective care. Clinical Decision Support Systems (CDSSs) show promise in enhancing care processes, but their effective implementation in self-harm management remains unexplored. Methods PERMANENS comprises a combination of methodologies and study designs aimed at developing a CDSS prototype that assists clinicians in the personalized assessment and management of ED patients presenting with self-harm. Ensemble prediction models will be constructed by applying machine learning techniques on electronic registry data from four sites, i.e., Catalonia (Spain), Ireland, Norway, and Sweden. These models will predict key adverse outcomes including self-harm repetition, suicide, premature death, and lack of post-discharge care. Available registry data include routinely collected electronic health record data, mortality data, and administrative data, and will be harmonized using the OMOP Common Data Model, ensuring consistency in terminologies, vocabularies and coding schemes. A clinical knowledge base of effective suicide prevention interventions will be developed rooted in a systematic review of clinical practice guidelines, including quality assessment of guidelines using the AGREE II tool. The CDSS software prototype will include a backend that integrates the prediction models and the clinical knowledge base to enable accurate patient risk stratification and subsequent intervention allocation. The CDSS frontend will enable personalized risk assessment and will provide tailored treatment plans, following a tiered evidence-based approach. Implementation research will ensure the CDSS’ practical functionality and feasibility, and will include periodic meetings with user-advisory groups, mixed-methods research to identify currently unmet needs in self-harm risk assessment, and small-scale usability testing of the CDSS prototype software. Discussion Through the development of the proposed CDSS software prototype, PERMANENS aims to standardize care, enhance clinician confidence, improve patient satisfaction, and increase treatment compliance. The routine integration of CDSS for self-harm risk assessment within healthcare systems holds significant potential in effectively reducing suicide mortality rates by facilitating personalized and timely delivery of effective interventions on a large scale for individuals at risk of suicide.

show abstract

The Veterans Health Administration REACH VET Program: Suicide Predictive Modeling in Practice

Cited by 10 publications

References 8 publications

Determining Distinct Suicide Attempts From Recurrent Electronic Health Record Codes: Classification Study

Determining Distinct Suicide Attempts From Recurrent Electronic Health Record Codes: Classification Study

Continuing the transformation: charting the path for the future delivery of Veteran emergency care

Developing a clinical decision support system software prototype that assists in the management of patients with self-harm in the emergency department: protocol of the PERMANENS project

Contact Info

Product

Resources

About