The Kaiser Permanente Northern California Advance Alert Monitor Program: An Automated Early Warning System for Adults at Risk for In-Hospital Clinical Deterioration

Martinez, Vanessa A.; Betts, Robin K.; Scruth, Elizabeth; Buckley, Jacqueline D.; Cadiz, Vilma R.; Bertrand, Linda D.; Paulson, Shirley; Dummett, B. Alex; Abhyankar, Stella S.; Reyes, Vivian; Hatton, Joeffrey R.; Sulit, Reynaldo; Vincent, Jean‐Louis

doi:10.1016/j.jcjq.2022.05.005

Cited by 6 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Effects on care-delivery were assessed using a variety of measures including time to treatment (e.g., [29,39,67,68]). Patient outcomes were assessed using measures like length of stay and mortality (e.g., [61,62,66]), but no studies examined adverse events due to AI errors. Though improvements in decision-making and care delivery are expected to improve patient outcomes, it cannot be assumed, making it essential to directly evaluate the effect of AI interventions on patient outcomes [1,16].…”

Section: Discussionmentioning

confidence: 99%

“…Three US studies examined assistive AI for detection of in-hospital deterioration. Martinez et al [61] described an early warning system that combined statistical modelling with ML to identify patients at risk of deterioration. Deployment across 19 hospitals was associated with decreases in mortality (10% vs. 14%), hospital length of stay, and intensive care unit length of stay.…”

Section: In-hospital Deteriorationmentioning

confidence: 99%

See 1 more Smart Citation

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

Magrabi,

Lyell,

Coiera

2023

Yearb Med Inform

View full text Add to dashboard Cite

Aims and objectives: To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings. Method: PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy. Reported effects on user experience, decision-making, care delivery and outcomes were summarised. Results: AI technologies are being applied in a wide variety of clinical areas. Most contemporary systems utilise deep learning, use routinely collected data, support diagnosis and triage, are assistive (requiring users to confirm or approve AI provided information or decisions), and are used by doctors in acute care settings in high-income nations. AI systems are integrated and used within existing clinical information systems including electronic medical records. There is limited support for One Health goals. Evaluation is largely based on quantitative methods measuring effects on decision-making. Conclusion: AI systems are being implemented and evaluated in many clinical areas. There remain many opportunities to understand patterns of routine use and evaluate effects on decision-making, care delivery and patient outcomes using mixed-methods. Support for One Health including integrating data about environmental factors and social determinants needs further exploration.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: In-hospital Deteriorationmentioning

confidence: 99%

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

Magrabi,

Lyell,

Coiera

2023

Yearb Med Inform

View full text Add to dashboard Cite

show abstract

“…Moreover, due to the limited sensitivity of EWS and the availability of continuous data, a crucial progression involves leveraging artificial intelligence, particularly machine learning and predictive algorithms, to enhance clinical decision support [75,76]. These analytics have the potential to identify specific patterns or 'signatures' of clinical deterioration before it becomes apparent, thereby transitioning from reactive (detection) to proactive care (prediction) [77]. Key considerations include aggregating vital signs [40], integrating contextual factors like signs/symptoms, patient circadian rhythm, and activity levels.…”

Section: Software Platforms For Trend Analysismentioning

confidence: 99%

Wearable wireless continuous vital signs monitoring on the general ward

Leenen,

Schoonhoven,

Patijn

2024

Current Opinion in Critical Care

View full text Add to dashboard Cite

Purpose of review Wearable wireless sensors for continuous vital signs monitoring (CVSM) offer the potential for early identification of patient deterioration, especially in low-intensity care settings like general wards. This study aims to review advances in wearable CVSM – with a focus on the general ward – highlighting the technological characteristics of CVSM systems, user perspectives and impact on patient outcomes by exploring recent evidence. Recent findings The accuracy of wearable sensors measuring vital signs exhibits variability, especially notable in ambulatory patients within hospital settings, and standard validation protocols are lacking. Usability of CMVS systems is critical for nurses and patients, highlighting the need for easy-to-use wearable sensors, and expansion of the number of measured vital signs. Current software systems lack integration with hospital IT infrastructures and workflow automation. Imperative enhancements involve nurse-friendly, less intrusive alarm strategies, and advanced decision support systems. Despite observed reductions in ICU admissions and Rapid Response Team calls, the impact on patient outcomes lacks robust statistical significance. Summary Widespread implementation of CVSM systems on the general ward and potentially outside the hospital seems inevitable. Despite the theoretical benefits of CVSM systems in improving clinical outcomes, and supporting nursing care by optimizing clinical workflow efficiency, the demonstrated effects in clinical practice are mixed. This review highlights the existing challenges related to data quality, usability, implementation, integration, interpretation, and user perspectives, as well as the need for robust evidence to support their impact on patient outcomes, workflow and cost-effectiveness.

show abstract

“…These tools often use vital signs and laboratory values in binary regression or machine learning classification models to predict whether a patient will deteriorate [ 2 ] However, implementation of these tools has often failed to lead to improved patient outcomes [ 3 ]. A successful example of translating model deployment into improved patient outcomes, the Advanced Alert Monitor, [ 4 ] combines predictions with dedicated surveillance teams and structured patient follow-up protocols, suggesting that careful selection of the response to model predictions is a crucial component of improving patient outcomes. Recent research of deterioration model implementation has suggested that aligning prediction model development with the proposed implementation pathway could further improve the impact of these models on clinical practice [ 5 ].…”

Section: Introductionmentioning

confidence: 99%

“…To do so, a probability threshold or “cutpoint” is used, above which to classify the patient as high risk. Cutpoints are often selected based on metrics including the sensitivity or specificity, but may also be selected based on the estimated number of alerts per ward per day, attempting to limit the total number of alerts to be within an acceptable range based on clinician workloads, [ 4 , 7 ] or based on the cost-effectiveness of the model-alert-response workflow [ 8 ]. These approaches are practical, but require the arbitrary dichotomisation of predicted risks.…”

Section: Introductionmentioning

confidence: 99%

Prioritising deteriorating patients using time-to-event analysis: prediction model development and internal–external validation

Blythe,

Parsons,

Barnett

et al. 2024

Crit Care

View full text Add to dashboard Cite

Background Binary classification models are frequently used to predict clinical deterioration, however they ignore information on the timing of events. An alternative is to apply time-to-event models, augmenting clinical workflows by ranking patients by predicted risks. This study examines how and why time-to-event modelling of vital signs data can help prioritise deterioration assessments using lift curves, and develops a prediction model to stratify acute care inpatients by risk of clinical deterioration. Methods We developed and validated a Cox regression for time to in-hospital mortality. The model used time-varying covariates to estimate the risk of clinical deterioration. Adult inpatient medical records from 5 Australian hospitals between 1 January 2019 and 31 December 2020 were used for model development and validation. Model discrimination and calibration were assessed using internal–external cross validation. A discrete-time logistic regression model predicting death within 24 h with the same covariates was used as a comparator to the Cox regression model to estimate differences in predictive performance between the binary and time-to-event outcome modelling approaches. Results Our data contained 150,342 admissions and 1016 deaths. Model discrimination was higher for Cox regression than for discrete-time logistic regression, with cross-validated AUCs of 0.96 and 0.93, respectively, for mortality predictions within 24 h, declining to 0.93 and 0.88, respectively, for mortality predictions within 1 week. Calibration plots showed that calibration varied by hospital, but this can be mitigated by ranking patients by predicted risks. Conclusion Time-varying covariate Cox models can be powerful tools for triaging patients, which may lead to more efficient and effective care in time-poor environments when the times between observations are highly variable.

show abstract

The Kaiser Permanente Northern California Advance Alert Monitor Program: An Automated Early Warning System for Adults at Risk for In-Hospital Clinical Deterioration

Cited by 6 publications

References 9 publications

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings

Wearable wireless continuous vital signs monitoring on the general ward

Prioritising deteriorating patients using time-to-event analysis: prediction model development and internal–external validation

Contact Info

Product

Resources

About