Development and validation of a prognostic COVID-19 severity assessment (COSA) score and machine learning models for patient triage at a tertiary hospital

Abstract

Background Clinical risk scores and machine learning models based on routine laboratory values could assist in automated early identification of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) patients at risk for severe clinical outcomes. They can guide patient triage, inform allocation of health care resources, and contribute to the improvement of clinical outcomes. Methods In- and out-patients tested positive for SARS-CoV-2 at the Insel Hospital Group Bern, Switzerland, between February 1st and August 31st (‘first wave’, n = 198) and September 1st through November 16th 2020 (‘second wave’, n = 459) were used as training and prospective validation cohort, respectively. A clinical risk stratification score and machine learning (ML) models were developed using demographic data, medical history, and laboratory values taken up to 3 days before, or 1 day after, positive testing to predict severe outcomes of hospitalization (a composite endpoint of admission to intensive care, or death from any cause). Test accuracy was assessed using the area under the receiver operating characteristic curve (AUROC). Results Sex, C-reactive protein, sodium, hemoglobin, glomerular filtration rate, glucose, and leucocytes around the time of first positive testing (− 3 to + 1 days) were the most predictive parameters. AUROC of the risk stratification score on training data (AUROC = 0.94, positive predictive value (PPV) = 0.97, negative predictive value (NPV) = 0.80) were comparable to the prospective validation cohort (AUROC = 0.85, PPV = 0.91, NPV = 0.81). The most successful ML algorithm with respect to AUROC was support vector machines (median = 0.96, interquartile range = 0.85–0.99, PPV = 0.90, NPV = 0.58). Conclusion With a small set of easily obtainable parameters, both the clinical risk stratification score and the ML models were predictive for severe outcomes at our tertiary hospital center, and performed well in prospective validation.

Machine Learning Based Clinical Decision Support System for Early COVID-19 Mortality Prediction

“…Different ML models have been proposed to predict risk of developing severe complications and mortality (15)(16)(17)(18)(19)(20)(21)(22)(23)(24)(25)(26)(27). This is important since there are limited resources compared to the increasing number of COVID-19 patients.…”

Identification of Variable Importance for Predictions of Mortality From COVID-19 Using AI Models for Ontario, Canada

“…A study by Schöning et al aiming to distinguish between severe and non-severe COVID-19 also found hypertension, along with diabetes mellitus (Type 2) and renal impairment to be prognostic of severe disease (12). Schöning et al used a variety of machine learning models which were trained using data from the first wave in Switzerland and externally validated using data from the second wave (confirming findings to be generalizable) and achieved an accuracy of AUC values ranging from 0.86 (decision tree induction) to 0.96 (support vector machine) (12). Kim et al used Korean National Health Insurance data to identify co-morbidities and factors that increase mortality using multivariate logistic regression analysis with a confidence interval of 95% (13).…”

“…with a 95% confidence interval (14). Both studies identified age, deprivation, diabetes, bronchitis and severe asthma as top risk factors for COVID-19 (12,13). Additionally, Kim et al found that dental disorders were associated with high co-morbidity risk (13).…”

Machine learning application for the prediction of SARS-CoV-2 infection using blood tests and chest radiograph

“…In terms of diagnostic capability with machine learning, some recent studies have also been performed, but with smaller datasets, lack of temporal validation and often without clinical comparison 24 – 26 . More recently several machine learning based approaches have been published demonstrating more broader applicability in COVID-19 related applications including triage assessment 27 , severity classifcaiton 28 , 29 , risk prognostication including mortality 30 as well as applying to multi-omics data 31 . For example, a similar approach was tried with similar findings also with an attempt for explanability similar to our study 32 .…”