Performance comparison of prediction models for neonatal sepsis using logistic regression, multiple discriminant analysis and artificial neural network

Abstract

Sepsis is one of the major causes of neonatal deaths worldwide. Majority of these neonatal deaths occurs in resource-poor countries due to inaccessibility of hospitals and absence of laboratories. Blood culture which is the gold standard to confirm sepsis is time-consuming and requires the presence of a laboratory. To start the antimicrobial therapy at the earliest, prediction models have been developed. A vast number of available prediction models require laboratory tests and cannot be used in the developing countries where such facilities do not exist. Therefore, there is a need for non-invasive prediction models. The objectives of this study are as follows: -(i) to train and test non-invasive prediction models for neonatal sepsis (ii) to compare the performance of the invasive with non-invasive prediction models. For this retrospective study, we extracted the data of 1446 neonates from the Medical Information Mart for Intensive Care (MIMIC) III data set. We trained and tested six prediction models using this data set. Three of these six models were trained using non-invasive parameters (model LR(NI), model ANN(NI) and model MDA (NI)) and three were trained using invasive and non-invasive parameters (model LR(O), model ANN(O) and model MDA(O)). The sensitivity of model LR(NI), model ANN(NI), model MDA(NI), model LR(O), model ANN(O) and model MDA(O ) at their optimum threshold values were 81.68%, 79.39%, 82.44%, 77.10%, 79.39% and 78.63% respectively. Whereas, specificity of the above mentioned models were 82.27%, 81.82%, 80.00%, 84.77%, 82.05% and 78.30% respectively. To decrease the neonatal mortality rate in resource-poor areas one may use non-invasive prediction models where invasive parameters are not available due to lack of resources, as shown by our study that non-invasive prediction models can achieve similar predictive capability as the invasive prediction models.

Neonatal sepsis prediction through clinical decision support algorithms: A systematic review

“…Sepsis was either defined as early onset sepsis (EOS, n = 1 study) 18 occurring within the first 3 days of life or as late‐onset sepsis (LOS, n = 26 studies) 5,6,16,17,19–24,26,30,31,33,34,36–46 with onset after 3–7 days of life. However, nine studies did not clearly specify timepoint of sepsis evaluation 15,25,27–29,32,35,47,48 . Infant age at sepsis diagnosis was only specified in four studies, 21,22,29,35 with all others only mentioning infant gestational age (GA) at birth or at inclusion in the study.…”

“…We included two publications derived from one RCT population, hereafter referred to as two RCTs for simplicity, and 34 nonrandomised studies. Overall, fifteen studies developed their own CDSA (machine learning and classical) and used a retrospective cohort for validation, [15][16][17][18][19][20][21][22][23][24][25][26][27][28][29] whilst ten studies developed their own CDSA (machine learning) and used a prospective cohort for validation. [30][31][32][33][34][35][36][37][38][39] Further, six studies performed algorithm testing using previously established classical algorithms in a retrospective cohort, 5,6,[40][41][42][43] and five studies in a prospective cohort.…”

“…However, nine studies did not clearly specify timepoint of sepsis evaluation. 15,25,[27][28][29]32,35,47,48 Infant age at sepsis diagnosis was only specified in four studies, 21,22,29,35 with all others only mentioning infant gestational age (GA) at birth or at inclusion in the study. Sepsis cases were either culture-confirmed (n = 25 studies) , 6,15,[17][18][19][20][21][22][24][25][26][27][28][29]32,33,36,[38][39][40][43][44][45][46][47] defined through clinician suspicion without need of a positive blood culture (n = 5 studies), 16,23,30,31,48 or both (n = 6 studies), 5,34,35,37,41,…”

“…We included 22 studies reporting on the usefulness of sepsis prediction via machine learning algorithms, including logistic regression, which learn internal parameters to perform the prediction task as accurately as possible. 6,[15][16][17]20,22,24,[27][28][29][30][31]34,[36][37][38][39]41,[43][44][45]47 Another 14 studies applied more classical algorithms, such as manually designed decision tree or hypothesis testing for relevant feature identifications, for sepsis prediction. 5,18,19,21,23,25,26,32,33,35,40,42,46,48 There was no significant difference in reported outcomes depending on the type of CDSA used, indicating that both applied techniques may be useful in the prediction of sepsis in infants.…”

“…The AUROC in detecting sepsis ranged from 0.63 to 0.88 with an average of 0.76. 6,15,16,20,22,24,27,28,31,34,36,40,41,44,45 Sensitivity, specificity, PPV, and NPV varied considerably between studies (Table 3).…”