In Reply In our study, 1 we pursued an exhaustive crossvalidated grid search to identify the optimal hyperparameters for the extreme gradient descent boosting model (XG-B oost), a standard approach to the selection of hyperparameters that included searching over the learning rate, number of trees trained, maximum tree depth, and minimum loss reduction required for partition on a leaf node on a tree. 2 To permit comparison of area under the receiver operator characteristic (AUROC) curves, we focused on defining their variance in iterative cross validation and reported as a 95% CI. Moreover, this approach allowed comparison of other metrics, such as the precision and recall, using a consistent approach for reporting confidence intervals. As reported in the study, XGBoost did not have better discrimination for inhospital mortality in acute myocardial infarction (AMI) than a logistic regression model (XGBoost: AUROC, 0.89; 95% CI, 0.88-0.89; logistic regression: AUROC, 0.88; 95% CI, 0.88-0.88) despite the large sample size and selection of optimal hyperparameters. 1 For the neural network, we chose a routinely used model architecture-a feedforward artificial neural network-that we then trained for predicting in-hospital mortality in AMI. 3 The network was composed of 5 fully connected hidden layers, each with 100 nodes and a rectified linear unit activation, and an output layer with a sigmoid activation function. While we could have built a more complex neural network by incorporating dropout layers and deeper, more complex architectures, our goal was to test models routinely used for structured data. Moreover, we leveraged the predictive power of individual models in a meta-classifier that was built on learnings from individual models. Therefore, the goal of our work was to evaluate the predictive gain over logistic regression in a series of routinely used models and a meta-learner, rather than iterative design of individual models.With respect to Pieszko and Slomka's question about models among patients with critical illness following AMI, our study proposal was approved for studying the broader population of patients presenting with AMI; models built for other subgroups, such as for those with cardiac arrest and cardiogenic shock, were outside the scope of our work.In sum, our findings are an accurate representation of the limited predictive gain from the use of machine learning in predictive models built with registry data. However, this result is not a verdict on machine learning but a reflection of models built on low-dimensional data that are currently manually abstracted into a set number of discrete fields, 4 as is currently done for clinical registries.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.