Stable prediction in high-dimensional linear models

Lin, Bingqing; Wang, Qihua; Zhang, Jun; Pang, Zhen

doi:10.1007/s11222-016-9694-6

Cited by 15 publications

(18 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In essence, StabSel is a general ensemble learning procedure which can be used in conjunction with any selection algorithm to boost its selection performance. Like many others , here we focus on the combination of StabSel with the lasso (abbreviated as SSLasso to facilitate later discussions), the most natural combination for high‐dimensional regression problems. In the generation stage, SSLasso applies the lasso repeatedly to subsamples randomly drawn from the training data.…”

Section: Ensemble Pruning Of Stability Selectionmentioning

confidence: 99%

“…During the review process, we were made aware of some recent works that also aim to re‐weigh individual ensemble members, for example, the “random splitting model averaging” (RSMA) procedure and the VSD‐selector (VSD‐S) which weighs individual ensemble members using a so‐called variable selection deviation metric . Both of these methods use weighting schemes that are much more elaborate than ours.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Pruning variable selection ensembles

Zhang

Zhu

2019

Statistical Analysis

View full text Add to dashboard Cite

In the context of variable selection, ensemble learning has gained increasing interest due to its great potential to improve selection accuracy and to reduce false discovery rate. A novel ordering-based selective ensemble learning strategy is designed in this paper to obtain smaller but more accurate ensembles. In particular, a greedy sorting strategy is proposed to rearrange the order by which the members are included into the integration process. Through stopping the fusion process early, a smaller subensemble with higher selection accuracy can be obtained. More importantly, the sequential inclusion criterion reveals the fundamental strength-diversity trade-off among ensemble members. By taking stability selection with the base learner lasso (abbreviated as SSLasso) as an example, some experiments are conducted to examine the performance of the novel algorithm. Experimental results demonstrate that pruned SSLasso generally achieves higher selection accuracy and lower false discovery rate than SSLasso and several other benchmark methods.

show abstract

Section: Ensemble Pruning Of Stability Selectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Pruning variable selection ensembles

Zhang

Zhu

2019

Statistical Analysis

View full text Add to dashboard Cite

show abstract

“…A specific case [26] of model-independent approach limited to linear models (with arbitrary solution algorithm and hyper-parameters) provides good results for heteroscedastic datasets ( [26], supplementary materials), and suits for ELM output layer solution as well. The method applies to any amount of training data, and will benefit from huge datasets by producing more independent models in its ensemble part.…”

Section: State-of-the-artmentioning

confidence: 99%

Per-sample prediction intervals for extreme learning machines

Akusok

Miché

Björk

et al. 2018

Int. J. Mach. Learn. & Cyber.

View full text Add to dashboard Cite

Prediction intervals in supervised Machine Learning bound the region where the true outputs of new samples may fall. They are necessary in the task of separating reliable predictions of a trained model from near random guesses, minimizing the rate of False Positives, and other problem-specific tasks in applied Machine Learning. Many real problems have heteroscedastic stochastic outputs, which explains the need of inputdependent prediction intervals.This paper proposes to estimate the input-dependent prediction intervals by a separate Extreme Learning Machine model, using variance of its predictions as a correction term accounting for the model uncertainty. The variance is estimated from the model's linear output layer with a weighted Jackknife method. The methodology is very fast, robust to heteroscedastic outputs, and handles both extremely large datasets and insufficient amount of training data.

show abstract

“…Thus, we increased the factor multiplying in Λ because is small in this simulation. We compared it with traditional stepwise search as well as some VSE techniques including BSS [20], PGA [18], RSMA [24], and ST2E [22]. The parameters involved in these methods were set according to the related literature.…”

Section: Simulation 3: Performance Comparison With Several Othermentioning

confidence: 99%

“…Another good property of stability selection is that it provides an effective way to control false discovery rate (FDR) in finite sample cases provided that its tuning parameters are set properly. Due to its versatility and flexibility, stability selection has been successfully applied in many domains such as gene expression analysis [24,[27][28][29]. Nevertheless, we have not found any literature about applying stability selection to a Cox model.…”

Section: Introductionmentioning

confidence: 99%

Ensembling variable selectors by stability selection for the Cox model

Zhang

2017

2017 International Conference on Machine Learning and Cybernetics (ICMLC)

View full text Add to dashboard Cite

As a pivotal tool to build interpretive models, variable selection plays an increasingly important role in high-dimensional data analysis. In recent years, variable selection ensembles (VSEs) have gained much interest due to their many advantages. Stability selection (Meinshausen and Bühlmann, 2010), a VSE technique based on subsampling in combination with a base algorithm like lasso, is an effective method to control false discovery rate (FDR) and to improve selection accuracy in linear regression models. By adopting lasso as a base learner, we attempt to extend stability selection to handle variable selection problems in a Cox model. According to our experience, it is crucial to set the regularization region Λ in lasso and the parameter min properly so that stability selection can work well. To the best of our knowledge, however, there is no literature addressing this problem in an explicit way. Therefore, we first provide a detailed procedure to specify Λ and min . Then, some simulated and real-world data with various censoring rates are used to examine how well stability selection performs. It is also compared with several other variable selection approaches. Experimental results demonstrate that it achieves better or competitive performance in comparison with several other popular techniques.

show abstract

Stable prediction in high-dimensional linear models

Cited by 15 publications

References 23 publications

Pruning variable selection ensembles

Pruning variable selection ensembles

Per-sample prediction intervals for extreme learning machines

Ensembling variable selectors by stability selection for the Cox model

Contact Info

Product

Resources

About