Early detection of in-patient deterioration: one prediction model does not fit all

Abstract

Objectives: Early detection of subacute potentially catastrophic illnesses using available data is a clinical imperative, and scores that report risk of imminent events in real time abound. Patients deteriorate for a variety of reasons, and it is unlikely that a single predictor such as an abnormal National Early Warning Score (NEWS) will detect all of them equally well. The objective of this study was to test the idea that the diversity of reasons for clinical deterioration leading to ICU transfer mandates multiple targeted predictive models. Design: Individual chart review to determine the clinical reason for ICU transfer; determination of relative risks of individual vital signs, lab tests and cardiorespiratory monitoring measures for prediction of each clinical reason for ICU transfer; logistic regression modeling for the outcome of ICU transfer for a specific clinical reason. Setting: Cardiac medical-surgical ward; tertiary care academic hospital. Patients: 8111 adult patients, 457 of whom were transferred to an ICU for clinical deterioration. Interventions: None. Measurements and main results: We calculated the contributing relative risks of individual vital signs, lab tests and cardiorespiratory monitoring measures for prediction of each clinical reason for ICU transfer, and used logistic regression modeling to calculate ROC areas and relative risks for the outcome of ICU transfer for a specific clinical reason. The reasons for clinical deterioration leading to ICU transfer were varied, as were their predictors. For example, the three most common reasons - respiratory instability, infection and suspected sepsis, and heart failure requiring escalated therapy - had distinct signatures of illness. Statistical models trained to target specific reasons for ICU transfer performed better than one model targeting combined events, and both performed better than the untrained NEWS score. Conclusions and relevance: A single predictive model for clinical deterioration does not perform as well as having multiple models trained for the individual specific clinical events leading to ICU transfer.

Principles for Real-World Implementation of Bedside Predictive Analytics Monitoring

“…10,11,13,15 To accept new tools like predictive analytics, particularly those that derive information from the physiologic waveforms, clinicians need to understand what is happening inside. 11,13,16,17 We presented the published evidence base 6,[18][19][20][21][22] to provide transparency into the algorithms' underpinnings and to emphasize the strengths of the scientific foundation. 16 We focused other sessions on current patients to give clinicians a more detailed examination of how data elements interacted within the algorithm to produce a risk score.…”

“…The predictive models are optimized to specific venues and target outcomes, differing from a standard "one-size-fits-all" approach). 21 (B) Bed 93 is selected from the Leaderboard. The bottom right graph displays the patient's CoMET scores as a function of time (24, 48, or 72 hours).…”

Causes, Diagnostic Testing, and Treatments Related to Clinical Deterioration Events among High-Risk Ward Patients

“…Although the predictors of clinical deterioration episodes, such as elevated respiratory rate, have been well described (11)(12)(13)(14), the medical conditions causing these episodes as well as their related diagnostic tests and treatments are poorly characterized. Prior epidemiological studies of clinical deterioration events have been limited to patients requiring certain interventions (e.g., ICU transfer or rapid response activation) or having specific vital sign triggers (e.g., elevated respiratory rate), measures which do not generalize to all deterioration events (15)(16)(17)(18). Because it is not the early warning scores but the process of diagnosis and treatment that they prompt that drives clinical improvement, an understanding of the most common causes of clinical deterioration would allow for the development of more targeted workflows.…”

Beyond prediction: Off‐target uses of artificial intelligence‐based predictive analytics in a learning health system

“…We have argued against this one‐size‐fits‐all approach, favoring the use of predictive models tailored for specific patient populations and target illnesses. 22 Because these models are specific to clinical events and physiologic systems, we suggest that clinicians can use them in ways that have the potential to promote optimal supportive care, responsiveness to therapy, and support overall resource allocation of a unit or hospital.…”

“…They were trained and tested on clinical deterioration events including sepsis, emergent intubation, emergent ICU transfer, bleeding, and others. 14 , 15 , 16 , 22 , 26 , 27 , 28 Unlike one‐size‐fits‐all models, they target specific clinical units and patient populations. The approach of AI‐based predictive analytics for early warning is based on the premise that there are often subtle changes that represent signatures of illness, or prodromes, that can be detected hours prior to an adverse clinical event.…”