Subset Selection in Regression

Miller, A.

doi:10.1201/9781420035933

Cited by 962 publications

(692 citation statements)

References 0 publications

Supporting

Mentioning

674

Contrasting

Unclassified

Order By: Relevance

“…This worked well and the performance reported here is comparable to those models 4 . It is interesting that the terms selected by the present method match, quite closely, those of the earlier models.…”

Section: Resultssupporting

confidence: 82%

“…This method is a bottom-up search procedure, where the term floating identifies that the number of features changes dynamically, with one feature included and/or excluded, at each iteration. A fairly comprehensive treatment of the question of subset selection [4] describes a number of other methods at some length and touches on the more recent focus on "datadriven" approaches, whereby a modified (regularized) objective function that in some way penalizes the inclusion of terms that have low value is optimized e.g. via a maximum likelihood procedure.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic covariate selection in logistic models for chest pain diagnosis: A new approach

Harrison

Kennedy

2008

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

A newly established method for optimizing logistic models via a minorizationmajorization procedure is applied to the problem of diagnosing acute coronary syndromes (ACS). The method provides a principled approach to the selection of covariates which would otherwise require the use of a suboptimal method owing to the size of the covariate set. A strategy for building models is proposed and two models optimized for performance and for simplicity are derived via ten-fold crossvalidation. These models confirm that a relatively small set of covariates including clinical and electrocardiographic features can be used successfully in this task.The performance of the models is comparable with previously published models using less principled selection methods. The models prove to be portable when tested on data gathered from three other sites. Whilst diagnostic accuracy and calibration diminishes slightly for these new settings, it remains satisfactory overall.The prospect of building predictive models that are as simple as possible for a required level of performance is valuable if data-driven decision aids are to gain wide acceptance in the clinical situation owing to the need to minimize the time taken to gather and enter data at the bedside.

show abstract

Section: Resultssupporting

confidence: 82%

Section: Introductionmentioning

confidence: 99%

Automatic covariate selection in logistic models for chest pain diagnosis: A new approach

Harrison

Kennedy

2008

Computer Methods and Programs in Biomedicine

View full text Add to dashboard Cite

show abstract

“…The research on sparse and regularized solutions has gained increasing interest during the last ten years [7]. This is partly due to advances in measurement technologies, e.g., in molecular biology, where high-throughput technologies allow simultaneous measurement of tens of thousands of variables.…”

Section: Methodsmentioning

confidence: 99%

Bioprocess data mining using regularized regression and random forests

Hassan

Farhan

Mangayil

et al. 2013

BMC Systems Biology

View full text Add to dashboard Cite

BackgroundIn bioprocess development, the needs of data analysis include (1) getting overview to existing data sets, (2) identifying primary control parameters, (3) determining a useful control direction, and (4) planning future experiments. In particular, the integration of multiple data sets causes that these needs cannot be properly addressed by regression models that assume linear input-output relationship or unimodality of the response function. Regularized regression and random forests, on the other hand, have several properties that may appear important in this context. They are capable, e.g., in handling small number of samples with respect to the number of variables, feature selection, and the visualization of response surfaces in order to present the prediction results in an illustrative way.ResultsIn this work, the applicability of regularized regression (Lasso) and random forests (RF) in bioprocess data mining was examined, and their performance was benchmarked against multiple linear regression. As an example, we used data from a culture media optimization study for microbial hydrogen production. All the three methods were capable in providing a significant model when the five variables of the culture media optimization were linearly included in modeling. However, multiple linear regression failed when also the multiplications and squares of the variables were included in modeling. In this case, the modeling was still successful with Lasso (correlation between the observed and predicted yield was 0.69) and RF (0.91).ConclusionWe found that both regularized regression and random forests were able to produce feasible models, and the latter was efficient in capturing the non-linearity in the data. In this kind of a data mining task of bioprocess data, both methods outperform multiple linear regression.

show abstract

“…In ordinary linear regression, there are a variety of classical techniques for variable selection such as forward selection, backward elimination and stepwise selection; see references [7,8] for a thorough review on these methods. To gauge the number of variables included in the model, Mallow's C p [9,10], Akaike's Information Criterion [11,12] and Schwartz's Bayesian Information Criterion [13] are widely used.…”

Section: Introductionmentioning

confidence: 99%

Variable selection for proportional odds model

Zhang

2007

Statistics in Medicine

View full text Add to dashboard Cite

SUMMARYIn this paper we study the problem of variable selection for the proportional odds model, which is a useful alternative to the proportional hazards model and might be appropriate when the proportional hazards assumption is not satisfied. We propose to fit the proportional odds model by maximizing the marginal likelihood subject to a shrinkage-type penalty, which encourages sparse solutions and hence facilitates the process of variable selection. Two types of shrinkage penalties are considered: the LASSO and the adaptive-LASSO penalty. In the adaptive-LASSO penalty, different weights are imposed on different coefficients such that important variables are more protectively retained in the final model while unimportant ones are more likely to be shrunk to zeros. We further provide an efficient computation algorithm to implement the proposed methods, and demonstrate their performance through simulation studies and an application to real data. Numerical results indicate that both methods can produce accurate and interpretable models, and the adaptive-LASSO tends to work better than the usual LASSO.

show abstract

Subset Selection in Regression

Cited by 962 publications

References 0 publications

Automatic covariate selection in logistic models for chest pain diagnosis: A new approach

Automatic covariate selection in logistic models for chest pain diagnosis: A new approach

Bioprocess data mining using regularized regression and random forests

Variable selection for proportional odds model

Contact Info

Product

Resources

About