On Oblique Random Forests

Menze, Bjoern; Kelm, B. Michael; Splitthoff, Daniel Nicolas; Koethe, Ullrich; Hamprecht, Fred A.

doi:10.1007/978-3-642-23783-6_29

Cited by 156 publications

(131 citation statements)

References 35 publications

Supporting

Mentioning

120

Contrasting

Unclassified

Order By: Relevance

“…RF typically refers to what Breiman called Forest-RIan ensemble of axis-parallel, or orthogonal decision trees [11,14]. In these types of trees, the feature space is recursively split along directions parallel to the axes of the feature space.…”

Section: Introductionmentioning

confidence: 99%

“…Thus, in cases in which the classes seem inseparable along any single dimension, RF may be suboptimal. To address this drawback, several variations of "oblique" decision forests have been proposed, including efforts to learn good projections [11,16], using principal components analysis to find the directions of maximal variance [12,15], or directly learning good discriminant directions [14]. Another recently proposed method, called Random Rotation RF (RR-RF), uniformly randomly rotates the data for every decision tree in the ensemble prior to inducing the tree [2].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ROFLMAO: Robust Oblique Forests with Linear MAtrix Operations

Tomita

Maggioni

Vogelstein

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Random Forest (RF) remains one of the most widely used general purpose classification methods. Two recent largescale empirical studies demonstrated it to be the best overall classification method among a variety of methods evaluated. One of its main limitations, however, is that it is restricted to only axis-aligned recursive partitions of the feature space. Consequently, RF is particularly sensitive to the orientation of the data. Several studies have proposed "oblique" decision forest methods to address this limitation. However, these methods either have a time and space complexity significantly greater than RF, are sensitive to unit and scale, or empirically do not perform as well as RF on real data. One promising oblique method that was proposed alongside the canonical RF method, called Forest-RC (F-RC), has not received as much attention by the community. Despite it being just as old as RF, virtually no studies exist investigating its theoretical or empirical performance. In this work, we demonstrate that F-RC empirically outperforms RF and another recently proposed oblique method called Random Rotation Random Forest, while approximately maintaining the same computational complexity. Furthermore, a variant of F-RC which rank transforms the data prior to learning is especially invariant to affine transformations and robust to data corruption. Open source code is available.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

ROFLMAO: Robust Oblique Forests with Linear MAtrix Operations

Tomita

Maggioni

Vogelstein

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

show abstract

“…While random forests are decision tree ensembles generated from orthogonal trees, oblique random forests are built from trees that split using linear discriminative models, such as lda, ridge regression and logistic regression. 68 The success of these models confirms the importance of including non-linear models in the prediction framework.…”

Section: Network-expression-mutation Signature Viewmentioning

confidence: 69%

Predicting colorectal cancer recurrence by utilizing multiple-view multiple-learner supervised learning.

Castellanos

Liu

Beauchamp

et al. 2017

JCO

View full text Add to dashboard Cite

635 Background: Colorectal cancer (CRC) remains a leading cause of cancer-related mortality in the United States. A key therapeutic dilemma in the treatment of CRC is whether patients with stage II and stage III disease require adjuvant chemotherapy after surgical resection. Attempts to improve identification of patients at increased risk of recurrence have yielded many predictive models based on gene expression data, but none are FDA approved and none are used in standard clinical practice. To improve recurrence prediction, we utilize a machine learning approach to predict recurrence status at 3 years after diagnosis. Methods: A dataset was curated from six publically available microarray datasets, and multiple views were generated to include information from non-tumor tissue gene expression patterns, gene set structure, protein-protein interaction network structure, previously curated molecular signatures, and identified tumor suppressor/driver mutations. These views were used to train a diverse pool of base learners using 10x 10-fold cross-validation. Stacked generalization was used to train an ensemble model, also known as a meta-learner, from the predictions of these base learners. Results: The performance of microarray trained models was significantly better compared to models trained on clinical data (Paired Wilcoxon signed rank test, p = 1.49 x 10-8), demonstrating that molecular data predicts recurrence significantly better than basic clinical data. Review of the model training performances revealed that non-linear classifiers often outperform linear classifiers, and that ensemble methods can also enhance performance. We also demonstrate the feasibility of the multiple-view multiple learner (MVML) supervised learning framework to generate and integrate predictions across a diverse set of learners, with the performance of the meta-learner exceeding or matching that of the best base learners across all performance metrics. Conclusions: This work represents the first effort to use ensemble learning to predict CRC recurrence and highlights the promise of ensemble learning to improve the performance of predictive models in order to realize the goals of precision medicine.

show abstract

“…In the context of high dimensional response (output) space, [15] is yet another interesting adaptation of random forest aimed at attaining every greater predictive performances. [1] and [17] have also recently proposed very interesting extension on the RF theme that seeks to further lower the prediction error. Coming from a perspective similar to ours, even though their base learners are still trees whereas we allow any type of base learner, [1] enriches the RF ensemble by way of a random subspace selection that gives less weight to weak features, i.e.…”

Section: Problem Formulationmentioning

confidence: 99%

Random Subspace Learning (RASSEL) with data driven weighting schemes

Fokoué¹,

Elshrif²

2018

Math. Appl.

View full text Add to dashboard Cite

Abstract. We present a novel adaptation of the random subspace learning approach to regression analysis and classification of high dimension low sample size data, in which the use of the individual strength of each explanatory variable is harnessed to achieve a consistent selection of a predictively optimal collection of base learners. In the context of random subspace learning, random forest (RF) occupies a prominent place as can be seen by the vast number of extensions of the random forest idea and the multiplicity of machine learning applications of random forest. The adaptation of random subspace learning presented in this paper differs from random forest in the following ways: (a) instead of using trees as RF does, we use multiple linear regression (MLR) as our regression base learner and the generalized linear model (GLM) as our classification base learner and (b) rather than selecting the subset of variables uniformly as RF does, we present the new concept of sampling variables based on a multinomial distribution with weights (success 'probabilities') driven through p independent one-way analysis of variance (ANOVA) tests on the predictor variables. The proposed framework achieves two substantial benefits, namely, (1) the avoidance of the extra computational burden brought by the permutations needed by RF to de-correlate the predictor variables, and (2) the substantial reduction in the average test error gained with the base learners used.

show abstract

On Oblique Random Forests

Cited by 156 publications

References 35 publications

ROFLMAO: Robust Oblique Forests with Linear MAtrix Operations

ROFLMAO: Robust Oblique Forests with Linear MAtrix Operations

Predicting colorectal cancer recurrence by utilizing multiple-view multiple-learner supervised learning.

Random Subspace Learning (RASSEL) with data driven weighting schemes

Contact Info

Product

Resources

About