A Machine Learning Approach for Mortality Prediction in COVID-19 Pneumonia: Development and Evaluation of the Piacenza Score

Abstract

Background Several models have been developed to predict mortality in patients with COVID-19 pneumonia, but only a few have demonstrated enough discriminatory capacity. Machine learning algorithms represent a novel approach for the data-driven prediction of clinical outcomes with advantages over statistical modeling. Objective We aimed to develop a machine learning–based score—the Piacenza score—for 30-day mortality prediction in patients with COVID-19 pneumonia. Methods The study comprised 852 patients with COVID-19 pneumonia, admitted to the Guglielmo da Saliceto Hospital in Italy from February to November 2020. Patients’ medical history, demographics, and clinical data were collected using an electronic health record. The overall patient data set was randomly split into derivation and test cohorts. The score was obtained through the naïve Bayes classifier and externally validated on 86 patients admitted to Centro Cardiologico Monzino (Italy) in February 2020. Using a forward-search algorithm, 6 features were identified: age, mean corpuscular hemoglobin concentration, PaO2/FiO2 ratio, temperature, previous stroke, and gender. The Brier index was used to evaluate the ability of the machine learning model to stratify and predict the observed outcomes. A user-friendly website was designed and developed to enable fast and easy use of the tool by physicians. Regarding the customization properties of the Piacenza score, we added a tailored version of the algorithm to the website, which enables an optimized computation of the mortality risk score for a patient when some of the variables used by the Piacenza score are not available. In this case, the naïve Bayes classifier is retrained over the same derivation cohort but using a different set of patient characteristics. We also compared the Piacenza score with the 4C score and with a naïve Bayes algorithm with 14 features chosen a priori. Results The Piacenza score exhibited an area under the receiver operating characteristic curve (AUC) of 0.78 (95% CI 0.74-0.84, Brier score=0.19) in the internal validation cohort and 0.79 (95% CI 0.68-0.89, Brier score=0.16) in the external validation cohort, showing a comparable accuracy with respect to the 4C score and to the naïve Bayes model with a priori chosen features; this achieved an AUC of 0.78 (95% CI 0.73-0.83, Brier score=0.26) and 0.80 (95% CI 0.75-0.86, Brier score=0.17), respectively. Conclusions Our findings demonstrated that a customizable machine learning–based score with a purely data-driven selection of features is feasible and effective for the prediction of mortality among patients with COVID-19 pneumonia.

How Artificial Intelligence and New Technologies Can Help the Management of the COVID-19 Pandemic

“…In all these cases, an ML model approximates a trained physician’s diagnosis with high accuracy [ 11 , 32 ]. ML can assist intensive care specialists in early detection of patient’s shock and septic conditions, including COVID-19 infections [ 33 ]. Epidemiologists may use ML forecasting capabilities in order to assess the spread of pandemics [ 34 , 35 , 36 ].…”

Artificial Intelligence in Digital Pathology: What Is the Future? Part 2: An Investigation on the Insiders

“…These studies focused on environmental measures; acquisition and sharing of knowledge in the general population and among clinicians; development and management of drugs and vaccines; remote psychological support of patients; remote monitoring, diagnosis, and follow-up; and maximization and rationalization of human and material resources in the hospital environment. The study described in [ 27 ] showed that AI-based scores with a purely data-driven selection of features are feasible and effective for the prediction of mortality among patients with COVID-19 pneumonia.…”

“…The three illustrated potentials [ 25 , 26 , 27 ] are also important in DP. In fact, in DP, the need for categorizing images merges with the need to make decisions and/or deduce approaches through actions on large databases and data sets or with other needs not based on medical images [ 1 , 19 , 20 ].…”

“…Therefore, the need and the justification of studies such as ours that tackle the introduction of AI focusing on acceptance and with a broad approach clearly emerges from these articles [ 25 , 26 , 27 ] .…”

Rapid assessment of COVID-19 mortality risk with GASS classifiers

“…The first COVID-19-specific scores were developed in China [43,44] 44,45 , uring the first wave of the pandemic. Further scores for prediction of COVID outcomes were later developed all over the world [12,45,46] 12,46,47 , and data from more heterogeneous datasets were collected in order to reduce potential selection bias stemming from sampling from a restricted population of subjects. A case in point is represented by the 4C score, developed by Knight et al [12] 12 : such a score has quickly become popular due to its vast derivation and validation cohorts and its ease of use.…”