The random forest (RF) algorithm has several hyperparameters that have to be set by the user, for example, the number of observations drawn randomly for each tree and whether they are drawn with or without replacement, the number of variables drawn randomly for each split, the splitting rule, the minimum number of samples that a node must contain, and the number of trees. In this paper, we first provide a literature review on the parameters' influence on the prediction performance and on variable importance measures. It is well known that in most cases RF works reasonably well with the default values of the hyperparameters specified in software packages. Nevertheless, tuning the hyperparameters can improve the performance of RF. In the second part of this paper, after a presenting brief overview of tuning strategies, we demonstrate the application of one of the most established tuning strategies, model‐based optimization (MBO). To make it easier to use, we provide the tuneRanger R package that tunes RF with MBO automatically. In a benchmark study on several datasets, we compare the prediction performance and runtime of tuneRanger with other tuning implementations in R and RF with default hyperparameters. This article is categorized under: Algorithmic Development > Biological Data Mining Algorithmic Development > Statistics Algorithmic Development > Hierarchies and Trees Technologies > Machine Learning

Background and goalThe Random Forest (RF) algorithm for regression and classification has considerably gained popularity since its introduction in 2001. Meanwhile, it has grown to a standard classification approach competing with logistic regression in many innovation-friendly scientific fields.ResultsIn this context, we present a large scale benchmarking experiment based on 243 real datasets comparing the prediction performance of the original version of RF with default parameters and LR as binary classification tools. Most importantly, the design of our benchmark experiment is inspired from clinical trial methodology, thus avoiding common pitfalls and major sources of biases.ConclusionRF performed better than LR according to the considered accuracy measured in approximately 69% of the datasets. The mean difference between RF and LR was 0.029 (95%-CI =[0.022,0.038]) for the accuracy, 0.041 (95%-CI =[0.031,0.053]) for the Area Under the Curve, and − 0.027 (95%-CI =[−0.034,−0.021]) for the Brier score, all measures thus suggesting a significantly better performance of RF. As a side-result of our benchmarking experiment, we observed that the results were noticeably dependent on the inclusion criteria used to select the example datasets, thus emphasizing the importance of clear statements regarding this dataset selection process. We also stress that neutral studies similar to ours, based on a high number of datasets and carefully designed, will be necessary in the future to evaluate further variants, implementations or parameters of random forests which may yield improved accuracy compared to the original version with default values.Electronic supplementary materialThe online version of this article (10.1186/s12859-018-2264-5) contains supplementary material, which is available to authorized users.

Multi-omics data, that is, datasets containing different types of high-dimensional molecular variables, are increasingly often generated for the investigation of various diseases. Nevertheless, questions remain regarding the usefulness of multi-omics data for the prediction of disease outcomes such as survival time. It is also unclear which methods are most appropriate to derive such prediction models. We aim to give some answers to these questions through a large-scale benchmark study using real data. Different prediction methods from machine learning and statistics were applied on 18 multi-omics cancer datasets (35 to 1000 observations, up to 100 000 variables) from the database ‘The Cancer Genome Atlas’ (TCGA). The considered outcome was the (censored) survival time. Eleven methods based on boosting, penalized regression and random forest were compared, comprising both methods that do and that do not take the group structure of the omics variables into account. The Kaplan–Meier estimate and a Cox model using only clinical variables were used as reference methods. The methods were compared using several repetitions of 5-fold cross-validation. Uno’s C-index and the integrated Brier score served as performance metrics. The results indicate that methods taking into account the multi-omics structure have a slightly better prediction performance. Taking this structure into account can protect the predictive information in low-dimensional groups—especially clinical variables—from not being exploited during prediction. Moreover, only the block forest method outperformed the Cox model on average, and only slightly. This indicates, as a by-product of our study, that in the considered TCGA studies the utility of multi-omics data for prediction purposes was limited. Contact:moritz.herrmann@stat.uni-muenchen.de, +49 89 2180 3198 Supplementary information: Supplementary data are available at Briefings in Bioinformatics online. All analyses are reproducible using R code freely available on Github.

Background:Soft-tissue sarcomas are a group of malignancies of mesenchymal origin, which typically have a dismal prognosis if they reach the metastatic stage. The observation of rare spontaneous remissions in patients suffering from concomitant bacterial infections had triggered the clinical investigation of the use of heat-killed bacteria as therapeutic agents (Coley's toxin), which induced complete responses in patients in the pre-chemotherapy era and is now known to mediate substantial elevations in serum TNF levels.Methods:We designed and developed a novel immunocytokine based on murine TNF sequentially fused to the antibody fragment F8 (specific to extra-domain A of fibronectin). The antitumor activity was studied in two syngeneic murine sarcoma models.Results:The L19 antibody (specific to extra-domain B of fibronectin) has shown by SPECT imaging procedures to selectively localise on sarcoma in a patient with a peripheral nerve sheath tumour, and immunohistochemical analysis of human soft-tissue sarcoma samples showed comparable antigen expression of EDA and EDB. The antibody-based pharmacodelivery of TNF by the fusion protein ‘F8–TNF' to oncofetal fibronectin in sarcoma-bearing mice leads to complete and long-lasting tumour eradications when administered in combination with doxorubicin, the first-line drug for the treatment of sarcomas in humans. Doxorubicin alone did not display any therapeutic effect in both tested models of this study. The cured mice had acquired protective immunity against the tumour, as they rejected subsequent challenges with sarcoma cells.Conclusion:The findings of this study provide a rationale for the clinical study of the fully human immunocytokine L19-TNF in combination with doxorubicin in patients with soft-tissue sarcoma.