LMIC-PRIEST: Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19 in a middle-income setting

Abstract

Background Uneven vaccination and less resilient health care systems mean hospitals in LMICs are at risk of being overwhelmed during periods of increased COVID-19 infection. Risk-scores proposed for rapid triage of need for admission from the emergency department (ED) have been developed in higher-income settings during initial waves of the pandemic. Methods Routinely collected data for public hospitals in the Western Cape, South Africa from the 27th August 2020 to 11th March 2022 were used to derive a cohort of 446,084 ED patients with suspected COVID-19. The primary outcome was death or ICU admission at 30 days. The cohort was divided into derivation and Omicron variant validation sets. We developed the LMIC-PRIEST score based on the coefficients from multivariable analysis in the derivation cohort and existing triage practices. We externally validated accuracy in the Omicron period and a UK cohort. Results We analysed 305,564 derivation, 140,520 Omicron and 12,610 UK validation cases. Over 100 events per predictor parameter were modelled. Multivariable analyses identified eight predictor variables retained across models. We used these findings and clinical judgement to develop a score based on South African Triage Early Warning Scores which also included age, sex, oxygen saturation, inspired oxygen, diabetes and heart disease. The LMIC-PRIEST score achieved C-statistics: 0.82 (95% CI: 0.82 to 0.83) development cohort; 0.79 (95% CI: 0.78 to 0.80) Omicron cohort; and 0.79 (95% CI: 0.79 to 0.80) UK cohort. Differences in prevalence of outcomes led to imperfect calibration in external validation. However, use of the score at thresholds of three or less would allow identification of very low-risk patients (NPV ≥0.99) who could be rapidly discharged using information collected at initial assessment. Conclusion The LMIC-PRIEST score shows good discrimination and high sensitivity at lower thresholds and can be used to rapidly identify low-risk patients in LMIC ED settings.

Training and testing of a gradient boosted machine learning model to predict adverse outcome in patients presenting to emergency departments with suspected covid-19 infection in a middle-income setting

“…Multivariable logistic regression with Least Absolute Shrinkage and Selection Operator (LASSO) and fractional polynomials achieved a slightly lower C-statistic of 0.87 (95% CI 0.866 to 0.874) and (CITL) of -0.017 (95%CI -0.043 to 0.009) on internal validation in Western Cape Alpha/Beta/Delta data, compared to the machine learning model reported here. [ 35 ] Discrimination on external validation was lower in the Omicron (C-statistic 0.79, 95% CI: 0.79 to 0.80) and Sudanese data (c-statistic 0.53, 95% CI: 0.53 to 0.54), but higher in the UK PRIEST, test data (C-statistic 0.79, 95% CI: 0.79 to 0.80). Machine learning is a trade-off between bias and variance, and despite the large sample size and use of regularization, there is a risk of over-fitting to the Western Cape data, which may explain the differential performance across test cohorts compared to a more conservative statistical modelling strategy.…”