edarf: Exploratory Data Analysis using Random Forests

Abstract

SummaryThis package contains functions useful for exploratory data analysis using random forests, which can be fit using the randomForest, randomForestSRC, or party packages (Liaw and Wiener 2002;Ishwaran and Kogalur 2013;Hothorn, Hornik, and Zeileis 2006). These functions can compute the partial dependence of covariates (individually or in combination) on the fitted forests' predictions, the permutation importance of covariates, as well as the distance between data points according to the fitted model.Random forests are an attractive method for social scientists. Random forests have only a few important tuning parameters and can be adapted to do classification, regression, and clustering. Many research tasks require interpretable models, and, although methods for interpreting random forests exist, in the most popular packages for fitting random forests these methods are available inconsistently. For example although there are methods for computing permutation importance in the randomForest, randomForestSRC and party packages, only randomForest can compute local importance. None of the packages can compute permutation importance for groups of covariates. Similarly partial dependence is only implemented in randomForest, and it has limited functionality compared to the functions provided herein. This software has been used in Jones and Linder (2015); Jones and Lupu (2016). Partial dependence, as described by Friedman (2001), estimates the marginal relationship between a subset of the covariates and the model's predictions by averaging over the marginal distribution of the compliment of this subset of the covariates. This approximation allows the display of the relationship between this subset of the covariates and the model's predictions even when there are many covariates which may interact. This functionality works with models fit by any of the aforementioned packages to any of the supported types of outcome variables. partial_dependence can be parallelized and also contains a number of additional parameters which allow the user to control this approximation. There is an associated plot function plot_pd which constructs plots for a wide variety of possible outputs from partial_dependence (e.g., when pairs of covariates are considered jointly, when each covariate is considered separately, when the outcome variable is categorical, etc).Permutation importance estimates the importance of a covariate by randomly shuffling its values, breaking any dependence between said covariate and the outcome, and then computing the difference between the predictions made by the model with that covariate shuffled and the predictions made when the covariate was not shuffled. If the covariate was useful in generating predictions then the prediction errors will increase in expectation when the covariate is shuffled, whereas no such increase can be expected when the covariate has no influence. Although all three of the random forest packages provide at least one method of assessing variable importance, variable_importance provides a c...

Random forest meteorological normalisation models for Swiss PM<sub>10</sub> trend analysis

“…Partial dependence plots allow RF models to be evaluated and to confirm how the explanatory variables are being used in the models for prediction (Jones and Linder, 2015). For the application presented here, there are general physical and chemical processes which should be confirmed in the RF models.…”

“…This allows RF to produce predictive models which generalise well and predictive performance is generally considered among the best of any ML technique (Caruana and Niculescu-Mizil, 2006). RF also has the advantage of not being a "black-box" method (Jones and Linder, 2015 the few ML techniques where the learning process can be explained, investigated, and interpreted. In the case of artificial neural networks or kernel based learning methods, this is much more difficult to do (Kotsiantis, 2013;Tong et al, 2003).…”

Random forest meteorological normalisation models for Swiss PM<sub>10</sub> trend analysis

“…Here, this advantage will be leveraged to 10 help explain some of the features in the PM 10 trends in Switzerland between 1997 and 2016. Partial dependence plots allow RF models to be evaluated and to confirm how the explanatory variables are being used in the models for prediction (Jones and Linder, 2015). For the application presented here, there are general physical and chemical processes which should be confirmed in the RF models.…”

“…This allows RF to produce predictive models 15 which generalise well and predictive performance is generally considered among the best of any ML technique (Caruana and Niculescu-Mizil, 2006). RF also has the advantage of not being a "black-box" method (Jones and Linder, 2015). Decision trees are one of the few ML techniques where the learning process can be explained, investigated, and interpreted.…”

What Drives State-Sponsored Violence?: Evidence from Extreme Bounds Analysis and Ensemble Learning Models

“…For discussion on these important differences in conceptualisation see (Straus 2007;Finkel and Straus 2012) inants of state-sponsored atrocities. Our approach is similar to Hegre and Sambanis (2006) seminal analysis on the causes of civil war onset, but we provide additional tests to verify whether complex interactions and nonlinearities are driving the statistical results (Bell 2015;Jones and Linder 2015;Jones and Lupu 2018;Muchlinski et al 2015). In conducting this analysis, we address three debates in the mass violence literature:…”