Demonstrating the consequences of learning missingness patterns in early warning systems for preventative health care: A novel simulation and solution

Gillies, Christopher E.; Taylor, Daniel; Cummings, Brandon C; Ansari, Sardar; Islim, Fadi; Kronick, Steven L.; Medlin, Richard; Ward, Kevin R.

doi:10.1016/j.jbi.2020.103528

Cited by 9 publications

(17 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The variables used as predictors were collected from the EHR and broadly included vital signs and physiologic observations, laboratory and metabolic values, and demographics. We selected specific features based on previous analysis [13]. Vital signs used in the model included heart rate, respiratory rate, pulse oximetry, Glasgow Coma Scale (GCS), urine output, and blood pressure.…”

Section: Predictorsmentioning

confidence: 99%

“…A full list of features is presented in Table S1 in Multimedia Appendix 1 alongside their respective median, IQR, and missingness rate. Variables centered on treatment (eg, medication administration) were largely excluded as, similar to the missingness flags described in Gillies et al [13], the scores generated by the model may be less generalizable and novel to the clinician as patterns of care change between diseases (eg, COVID-19) or institutions. Multimedia Appendix 1 Table S2 describes the effects of including medications as features in more detail.…”

Section: Predictorsmentioning

confidence: 99%

“…In this study, we have applied our previously described model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to a cohort of patients testing positive for COVID-19 [13]. Initially developed to predict patient deterioration in the general wards, we have retrained the model to target those outcomes considered most relevant to the COVID-19 pandemic: ICU level of care, mechanical ventilation, and death.…”

Section: Introductionmentioning

confidence: 99%

“…PICTURE, like the EDI, was trained and tuned on a large non-COVID-19 cohort including patients both with and without infectious diseases (131,546 encounters). Furthermore, we took extensive steps in the PICTURE framework to limit overfitting and learning missingness patterns in the data, such as a novel imputation mechanism [13]. This is critical in providing clinicians with novel, useful, and generalizable alerts, as missing patterns can vary in different settings and different patient phenotypes [13].…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, we took extensive steps in the PICTURE framework to limit overfitting and learning missingness patterns in the data, such as a novel imputation mechanism [13]. This is critical in providing clinicians with novel, useful, and generalizable alerts, as missing patterns can vary in different settings and different patient phenotypes [13]. In addition to the risk score, PICTURE also provides actionable explanations for its predictions in the form of Shapley values, which may help clinicians easily interpret scores and better determine if actionability on the alert is required [14].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods

Cummings¹,

Ansari²,

Motyka³

et al. 2021

JMIR Med Inform

Self Cite

View full text Add to dashboard Cite

Background: COVID-19 has led to an unprecedented strain on health care facilities across the United States. Accurately identifying patients at an increased risk of deterioration may help hospitals manage their resources while improving the quality of patient care. Here, we present the results of an analytical model, Predicting Intensive Care Transfers and Other Unforeseen Events (PICTURE), to identify patients at high risk for imminent intensive care unit transfer, respiratory failure, or death, with the intention to improve the prediction of deterioration due to COVID-19.Objective: This study aims to validate the PICTURE model's ability to predict unexpected deterioration in general ward and COVID-19 patients, and to compare its performance with the Epic Deterioration Index (EDI), an existing model that has recently been assessed for use in patients with COVID-19. Methods:The PICTURE model was trained and validated on a cohort of hospitalized non-COVID-19 patients using electronic health record data from 2014 to 2018. It was then applied to two holdout test sets: non-COVID-19 patients from 2019 and patients testing positive for COVID-19 in 2020. PICTURE results were aligned to EDI and NEWS scores for head-to-head comparison via area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve. We compared the models' ability to predict an adverse event (defined as intensive care unit transfer, mechanical ventilation use, or death). Shapley values were used to provide explanations for PICTURE predictions.

show abstract

Section: Predictorsmentioning

confidence: 99%

Section: Predictorsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods

Cummings¹,

Ansari²,

Motyka³

et al. 2021

JMIR Med Inform

Self Cite

View full text Add to dashboard Cite

show abstract

Deep anonymization of medical imaging

Fezai

Urruty

Bourdon

et al. 2022

Multimed Tools Appl

View full text Add to dashboard Cite

Outlook of pandemic preparedness in a post-COVID-19 world

Williams,

Jones,

Welch

et al. 2023

npj Vaccines

View full text Add to dashboard Cite

The COVID-19 pandemic was met with rapid, unprecedented global collaboration and action. Even still, the public health, societal, and economic impact may be felt for years to come. The risk of another pandemic occurring in the next few decades is ever-present and potentially increasing due to trends such as urbanization and climate change. While it is difficult to predict the next pandemic pathogen threat, making reasonable assumptions today and evaluating prior efforts to plan for and respond to disease outbreaks and pandemics may enable a more proactive, effective response in the future. Lessons from the COVID-19 response and pandemic influenza preparedness underscore the importance of strengthening surveillance systems, investing in early-stage research on pandemic pathogens and development of platform technologies, and diversifying response plans across a range of tactics to enable earlier access to safe and effective interventions in the next pandemic. Further, sustaining the robust vaccine manufacturing capacity built because of COVID-19 will keep it ready for rapid response in the future. These actions will not be successful without improved global coordination and collaboration. Everyone, including the biopharmaceutical industry, has a role to play in pandemic preparedness, and working together will ensure that the most lives are saved in the next pandemic.

show abstract

Demonstrating the consequences of learning missingness patterns in early warning systems for preventative health care: A novel simulation and solution

Abstract: doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.

Cited by 9 publications

References 32 publications

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods

Predicting Intensive Care Transfers and Other Unforeseen Events: Analytic Model Validation Study and Comparison to Existing Methods

Deep anonymization of medical imaging

Outlook of pandemic preparedness in a post-COVID-19 world

Contact Info

Product

Resources

About