Model Selection and Minimax Estimation in Generalized Linear Models

Abramovich, Felix; Grinshtein, Vadim

doi:10.1109/tit.2016.2555812

Cited by 21 publications

(18 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although logistic regression is widely used in various classification problems, its rigorous theoretical ground has not been yet properly established. Model selection in a general framework of generalized linear models (GLM) and in logistic regression in particular was studied in Abramovich and Grinshtein (2016). They proposed model selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and investigated the goodness-of-fit of the resulting estimator in terms of the Kullback-Leibler risk.…”

Section: Introductionmentioning

confidence: 99%

High-Dimensional Classification by Sparse Logistic Regression

Abramovich

Grinshtein

2019

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

We consider high-dimensional binary classification by sparse logistic regression. We propose a model/feature selection procedure based on penalized maximum likelihood with a complexity penalty on the model size and derive the non-asymptotic bounds for its misclassification excess risk. To assess its tightness we establish the corresponding minimax lower bounds. The bounds can be reduced under the additional low-noise condition. The proposed complexity penalty is remarkably related to the VC-dimension of a set of sparse linear classifiers. Implementation of any complexity penalty-based criterion, however, requires a combinatorial search over all possible models. To find a model selection procedure computationally feasible for high-dimensional data, we extend the Slope estimator for logistic regression and show that under an additional weighted restricted eigenvalue condition it is rate-optimal in the minimax sense.

show abstract

Section: Introductionmentioning

confidence: 99%

High-Dimensional Classification by Sparse Logistic Regression

Abramovich

Grinshtein

2019

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

show abstract

“…The generalized logistic regression analysis model also contains a random component with the Bernoulli distribution to characterize the stochastic effects [21]. The logit link function of generalized logistic regression analysis calculates the natural logarithm of an odds ratio of the binomial probabilities, which can be written as

\begin{matrix} T1 \\ l n (\frac{P_{I P D}}{P_{C O}}) = x^{T} β = β_{0} + x_{1} β_{1} + \dots + x_{f} β_{f}, \end{matrix}

where P IPD and P CO = 1 − P IPD denote the probabilities of binary classes (i.e., IPD and CO subject groups), the vector β = [ β 0 , β 1 ,…, β f ] T represents the generalized logistic regression coefficients, and x = [1, x 1 ,…, x f ] T is the model input vector including unity and five selected vocal features, the latter of which include MDVP:F0, Spread1, MDVP-LDA, Shimmer-LDA, and Nonlinear-LDA.…”

Section: Methodsmentioning

confidence: 99%

Dysphonic Voice Pattern Analysis of Patients in Parkinson’s Disease Using Minimum Interclass Probability Risk Feature Selection and Bagging Ensemble Learning Methods

Chen

Yao

et al. 2017

Computational and Mathematical Methods in Medicine

View full text Add to dashboard Cite

Analysis of quantified voice patterns is useful in the detection and assessment of dysphonia and related phonation disorders. In this paper, we first study the linear correlations between 22 voice parameters of fundamental frequency variability, amplitude variations, and nonlinear measures. The highly correlated vocal parameters are combined by using the linear discriminant analysis method. Based on the probability density functions estimated by the Parzen-window technique, we propose an interclass probability risk (ICPR) method to select the vocal parameters with small ICPR values as dominant features and compare with the modified Kullback-Leibler divergence (MKLD) feature selection approach. The experimental results show that the generalized logistic regression analysis (GLRA), support vector machine (SVM), and Bagging ensemble algorithm input with the ICPR features can provide better classification results than the same classifiers with the MKLD selected features. The SVM is much better at distinguishing normal vocal patterns with a specificity of 0.8542. Among the three classification methods, the Bagging ensemble algorithm with ICPR features can identify 90.77% vocal patterns, with the highest sensitivity of 0.9796 and largest area value of 0.9558 under the receiver operating characteristic curve. The classification results demonstrate the effectiveness of our feature selection and pattern analysis methods for dysphonic voice detection and measurement.

show abstract

“…For most models belonging the canonical exponential family, the step III. (ii) is quite trivial, see Lemma 1 in Abramovich and Grinshtein (2016) for example. Nonetheless, it is worthy to note that the our loss function is not in the canonical exponential family, so there is no extent discussion about the lower bound of likelihood-based divergence of θ and θ * in our setting.…”

Section: (Iii) the Last Term In (A2)mentioning

confidence: 99%

Heterogeneous Overdispersed Count Data Regressions via Double Penalized Estimations

Li¹,

Wei²,

Lei³

2021

Preprint

View full text Add to dashboard Cite

This paper studies the non-asymptotic merits of the double ℓ1-regularized for heterogeneous overdispersed count data via negative binomial regressions. Under the restricted eigenvalue conditions, we prove the oracle inequalities for Lasso estimators of two partial regression coefficients for the first time, using concentration inequalities of empirical processes. Furthermore, derived from the oracle inequalities, the consistency and convergence rate for the estimators are the theoretical guarantees for further statistical inference. Finally, both simulations and a real data analysis demonstrate that the new methods are effective.

show abstract

Model Selection and Minimax Estimation in Generalized Linear Models

Cited by 21 publications

References 26 publications

High-Dimensional Classification by Sparse Logistic Regression

High-Dimensional Classification by Sparse Logistic Regression

Dysphonic Voice Pattern Analysis of Patients in Parkinson’s Disease Using Minimum Interclass Probability Risk Feature Selection and Bagging Ensemble Learning Methods

Heterogeneous Overdispersed Count Data Regressions via Double Penalized Estimations

Contact Info

Product

Resources

About