Making complex prediction rules applicable for readers: Current practice in random forest literature and recommendations

Boulesteix, Anne‐Laure; Janitza, Silke; Hornung, Roman; Probst, Philipp; Busen, Hannah; Hapfelmeier, Alexander

doi:10.1002/bimj.201700243

Cited by 17 publications

(6 citation statements)

References 28 publications

(47 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Methods yielding models which are sparse with regard to the number of variables and number of omics types are often considered preferable from a practical perspective. Interpretation and practical application of the model to the prediction of independent data are easier with regression-based methods yielding coefficients that reflect the effects of variables on the outcome than with machine learning algorithms [ 8 ].…”

Section: Methodsmentioning

confidence: 99%

Large-scale benchmark study of survival prediction methods using multi-omics data

Herrmann

Probst

Hornung

et al. 2020

Briefings in Bioinformatics

Self Cite

152

View full text Add to dashboard Cite

Multi-omics data, that is, datasets containing different types of high-dimensional molecular variables, are increasingly often generated for the investigation of various diseases. Nevertheless, questions remain regarding the usefulness of multi-omics data for the prediction of disease outcomes such as survival time. It is also unclear which methods are most appropriate to derive such prediction models. We aim to give some answers to these questions through a large-scale benchmark study using real data. Different prediction methods from machine learning and statistics were applied on 18 multi-omics cancer datasets (35 to 1000 observations, up to 100 000 variables) from the database ‘The Cancer Genome Atlas’ (TCGA). The considered outcome was the (censored) survival time. Eleven methods based on boosting, penalized regression and random forest were compared, comprising both methods that do and that do not take the group structure of the omics variables into account. The Kaplan–Meier estimate and a Cox model using only clinical variables were used as reference methods. The methods were compared using several repetitions of 5-fold cross-validation. Uno’s C-index and the integrated Brier score served as performance metrics. The results indicate that methods taking into account the multi-omics structure have a slightly better prediction performance. Taking this structure into account can protect the predictive information in low-dimensional groups—especially clinical variables—from not being exploited during prediction. Moreover, only the block forest method outperformed the Cox model on average, and only slightly. This indicates, as a by-product of our study, that in the considered TCGA studies the utility of multi-omics data for prediction purposes was limited. Contact:moritz.herrmann@stat.uni-muenchen.de, +49 89 2180 3198 Supplementary information: Supplementary data are available at Briefings in Bioinformatics online. All analyses are reproducible using R code freely available on Github.

show abstract

Section: Methodsmentioning

confidence: 99%

Large-scale benchmark study of survival prediction methods using multi-omics data

Herrmann

Probst

Hornung

et al. 2020

Briefings in Bioinformatics

Self Cite

152

View full text Add to dashboard Cite

show abstract

“…RF is a regression-based classification algorithm that aggregates many decision trees trained on randomly sampled subsets of a complex dataset. 24 , 25 When developing the RF model, a grid search was used to determine the best combination of tuning parameters, including number of trees and number of features at each split. These models were chosen based on previous machine learning studies that focused on binary classifications.…”

Section: Methodsmentioning

confidence: 99%

Prediction of suitable outpatient candidates following revision total knee arthroplasty using machine learning

et al. 2023

View full text Add to dashboard Cite

AimsTo identify variables independently associated with same-day discharge (SDD) of patients following revision total knee arthroplasty (rTKA) and to develop machine learning algorithms to predict suitable candidates for outpatient rTKA.MethodsData were obtained from the American College of Surgeons National Quality Improvement Programme (ACS-NSQIP) database from the years 2018 to 2020. Patients with elective, unilateral rTKA procedures and a total hospital length of stay between zero and four days were included. Demographic, preoperative, and intraoperative variables were analyzed. A multivariable logistic regression (MLR) model and various machine learning techniques were compared using area under the curve (AUC), calibration, and decision curve analysis. Important and significant variables were identified from the models.ResultsOf the 5,600 patients included in this study, 342 (6.1%) underwent SDD. The random forest (RF) model performed the best overall, with an internally validated AUC of 0.810. The ten crucial factors favoring SDD in the RF model include operating time, anaesthesia type, age, BMI, American Society of Anesthesiologists grade, race, history of diabetes, rTKA type, sex, and smoking status. Eight of these variables were also found to be significant in the MLR model.ConclusionThe RF model displayed excellent accuracy and identified clinically important variables for determining candidates for SDD following rTKA. Machine learning techniques such as RF will allow clinicians to accurately risk-stratify their patients preoperatively, in order to optimize resources and improve patient outcomes.Cite this article: Bone Jt Open 2023;4(6):399–407.

show abstract

“… 15 Random forest is a regression-based classification algorithm with the aggregation of a large number of decision trees trained on randomly sampled subsets of complex dataset. 16 , 17 Random forest applies randomness when building each individual tree, and thus prediction by grouping could be more accurate than any individual tree. Both models were trained and tested using cross-validation with 80% data as a training set and 20% as a validation set.…”

Section: Methodsmentioning

confidence: 99%

Machine Learning Approaches to Define Candidates for Ambulatory Single Level Laminectomy Surgery

Zhong

Girardi

et al. 2021

Global Spine Journal

View full text Add to dashboard Cite

Study Design: retrospective cohort study. Objectives: To test and compare 2 machine learning algorithms to define characteristics associated with candidates for ambulatory same day laminectomy surgery. Methods: The American College of Surgeons National Surgical Quality Improvement Program database was queried for patients who underwent single level laminectomy in 2017 and 2018. The main outcome was ambulatory same day discharge. Study variables of interest included demographic information, comorbidities, preoperative laboratory values, and intra-operative information. Two machine learning predictive modeling algorithms, artificial neural network (ANN) and random forest, were trained to predict same day discharge. The quality of models was evaluated with area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) measures. Results: Among 35,644 patients, 13,230 (37.1%) were discharged on the day of surgery. Both ANN and RF demonstrated a satisfactory model quality in terms of AUC (0.77 and 0.77), accuracy (0.69 and 0.70), sensitivity (0.83 and 0.58), specificity (0.55 and 0.80), PPV (0.77 and 0.69), and NPV (0.64 and 0.70). Both models highlighted several important predictive variables, including age, duration of operation, body mass index and preoperative laboratory values including, hematocrit, platelets, white blood cells, and alkaline phosphatase. Conclusion: Machine learning approaches provide a promising tool to identify candidates for ambulatory laminectomy surgery. Both machine learning algorithms highlighted the as yet unrecognized importance of preoperative laboratory testing on patient pathway design.

show abstract

Making complex prediction rules applicable for readers: Current practice in random forest literature and recommendations

Cited by 17 publications

References 28 publications

Large-scale benchmark study of survival prediction methods using multi-omics data

Large-scale benchmark study of survival prediction methods using multi-omics data

Prediction of suitable outpatient candidates following revision total knee arthroplasty using machine learning

Machine Learning Approaches to Define Candidates for Ambulatory Single Level Laminectomy Surgery

Contact Info

Product

Resources

About