Controlling false discoveries in high-dimensional situations: boosting with stability selection

Abstract

BackgroundModern biotechnologies often result in high-dimensional data sets with many more variables than observations (n≪p). These data sets pose new challenges to statistical analysis: Variable selection becomes one of the most important tasks in this setting. Similar challenges arise if in modern data sets from observational studies, e.g., in ecology, where flexible, non-linear models are fitted to high-dimensional data. We assess the recently proposed flexible framework for variable selection called stability selection. By the use of resampling procedures, stability selection adds a finite sample error control to high-dimensional variable selection procedures such as Lasso or boosting. We consider the combination of boosting and stability selection and present results from a detailed simulation study that provide insights into the usefulness of this combination. The interpretation of the used error bounds is elaborated and insights for practical data analysis are given.ResultsStability selection with boosting was able to detect influential predictors in high-dimensional settings while controlling the given error bound in various simulation scenarios. The dependence on various parameters such as the sample size, the number of truly influential variables or tuning parameters of the algorithm was investigated. The results were applied to investigate phenotype measurements in patients with autism spectrum disorders using a log-linear interaction model which was fitted by boosting. Stability selection identified five differentially expressed amino acid pathways.ConclusionStability selection is implemented in the freely available R package stabs (http://CRAN.R-project.org/package=stabs). It proved to work well in high-dimensional settings with more predictors than observations for both, linear and additive models. The original version of stability selection, which controls the per-family error rate, is quite conservative, though, this is much less the case for its improvement, complementary pairs stability selection. Nevertheless, care should be taken to appropriately specify the error bound.Electronic supplementary materialThe online version of this article (doi:10.1186/s12859-015-0575-3) contains supplementary material, which is available to authorized users.

Development and evaluation of a multimodal marker of major depressive disorder

“…To determine whether feature selection is sensitive to choice of data reduction procedure, we also performed the MDD/control and continuous prediction without feature selection and obtained similar results. Further, a recently proposed variable selection algorithm, stability selection, was also applied (data not shown) (Hofner, Boccuto, & Goker, 2015; Hofner & Hothorn, 2017; Meinshausen & Bühlmann, 2010; Shah & Samworth, 2013). All features except one selected by this method for different outcomes fall within 39 features in Table A1.…”

Personality Traits Predict Smartphone Usage

“…Furthermore, this procedure allows for the assessment of the selection stability of variables while controlling for sample error. Hofner et al () suggests the upper limit of the pairwise family error rate (PFER) to be set at α < PFER max < mα ,, where m represents the number of predictors and α represents the respective significance level ( m factor α = 9 × 0.05 = 0.45 and m facet α ) =34 × 0.05 = 1.7 in our case). Based on this recommendation, we used an even lower PFER of 0.20.…”

Modeling spatiotemporal abundance of mobile wildlife in highly variable environments using boosted GAMLSS hurdle models

“…We fitted GAM and GAMLSS using an iterative machine‐learning approach, component‐wise functional gradient descent boosting (Bühlmann & Hothorn, ; Hothorn et al, ; Mayr et al, ; Hofner, Boccuto, & Göker, ; Mayr & Hofner, ) in a cyclical framework (Thomas et al, ). The first step of this process was to compute the negative gradient of a pre‐selected loss function, which acts as a working residual by giving more weight to observations not properly predicted in previous iterations.…”