Background
Global Registry of Acute Coronary Events (GRACE) score is a well-known model used to predict the probability of events in acute coronary syndrome (ACS). GRACE model was developed using a logistic regression approach that can only model linear functions, a limitation that could be prevented using artificial neural networks (NN) a recognized tool for nonlinear statistical modeling. The aim of this study was to develop, train and test different NN algorithm-based models to improve the GRACE score performance.
Methods
We analyzed a prospective database including 1,255 patients admitted with diagnosis of ACS in a community hospital, between June 2008 and June 2017. The database included 40 demographic and laboratory admission variables. In the guided approach, only the individual predictors included in the GRACE score were used to train and test three NN algorithm-based models, one- and two-hidden layer multilayer perceptron (MLP), and a radial basis function network. In addition, three extra unguided models were built using the 40 admission variables. Finally, expected mortality according to the GRACE score was calculated using the logistic regression equation.
The database was split into 2 datasets: 70% for model training and 30% for validation. In order to choose the best model, the training process was repeated 50 times. Every time the models were tested on the validation cohort, accuracy, receiver operating characteristic (ROC) area, negative predictive value (NPV), and positive predictive value (PPV) were recorded. Only models showing the best discrimination power were selected for comparison with logistic regression outcomes. The end point was in-hospital all-cause mortality.
Results
In terms of accuracy, ROC area and NPV, almost all NN algorithms outperformed the logistic regression approach (accuracy 97.1, 96.7, 96.2, 97.3 and 94.1%, p<0.001; ROC area 0.89, 0.86, 0.84, 0.84 and 0.75, Hanley-McNeil p≤0.05; for guided and unguided one- and two-hidden layers MLP and GRACE score, respectively). Only radial basis function models obtained a better accuracy level based on NPV improvement (100 vs. 98.8%, p=0.0001), at the expense of PPV reduction (0.0% vs. 13.2%, p<0.0001) (ROC are 0.84 vs. 0.75, p=0.043). Compared with the logistic regression approach, one- and two-hidden layers in guided and unguided MLP models improved PPV from 13.2 to 18.2% (38% increase), 15.4% (17% increase), 27.3% (107% increase), and 25.0% (89% increase), respectively, although these differences were not statistically significant.
Conclusions
NN algorithms improve GRACE score performance in terms of discriminatory power for the prediction of in-hospital mortality. Its application should become a useful tool for the decision making in ACS patients
