Oracle inequalities for sign constrained generalized linear models

Koike, Yuta; Tanoue, Yuta

doi:10.1016/j.ecosta.2019.02.001

Cited by 6 publications

(4 citation statements)

References 32 publications

(77 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To begin with, we first consider linear equality constrained model for a multiple linear model and then extend the framework to generalized linear model (GLM). Later, we show through numerical illustrations that the prior on the constrained space acts as a 'natural penalty' for the high-dimensional case (the so-called p ≥ n case) for multiple linear model and this feature is similar in spirit to the work by Koike and Tanoue (2019) within the frequentist framework which uses only non-negativity constraints to produce naturally sparse estimators.…”

Section: Bayesian Models For Constrained Parametersmentioning

confidence: 84%

“…In recent years, in the era of high-dimensional models, sign-constrained least square estimators are shown to provide sparse recovery without any regularization (e.g., see Meinshausen (2013) and Slawski and Hein (2013)) and such non-negatively constrained models have also been developed for generalized linear model (see Koike and Tanoue (2019)). This has been a remarkable development in the era of big data which allows for the incorporation of background information to obtain sparse solution instead of generic penalty functions.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Inference for Generalized Linear Model with Linear Inequality Constraints

Ghosal,

Ghosh

2021

Preprint

View full text Add to dashboard Cite

Bayesian statistical inference for Generalized Linear Models (GLMs) with parameters lying on a constrained space is of general interest (e.g., in monotonic or convex regression), but often constructing valid prior distributions supported on a subspace spanned by a set of linear inequality constraints can be challenging, especially when some of the constraints might be binding leading to a lower dimensional subspace. For the general case with canonical link, it is shown that a generalized truncated multivariate normal supported on a desired subspace can be used. Moreover, it is shown that such prior distribution facilitates the construction of a general purpose product slice sampling method to obtain (approximate) samples from corresponding posterior distribution, making the inferential method computationally efficient for a wide class of GLMs with an arbitrary set of linear inequality constraints. The proposed product slice sampler is shown to be uniformly ergodic, having a geometric convergence rate under a set of mild regularity conditions satisfied by many popular GLMs (e.g., logistic and Poisson regressions with constrained coefficients). One of the primary advantages of the proposed Bayesian estimation method over classical methods is that uncertainty of parameter estimates is easily quantified by using the samples simulated from the path of the Markov Chain of the slice sampler. Numerical illustrations using simulated data sets are presented to illustrate the superiority of the proposed methods compared to some existing methods in terms of sampling bias and variances. In addition, real case studies are presented using data sets for fertilizercrop production and estimating the SCRAM rate in nuclear power plants.

show abstract

Section: Bayesian Models For Constrained Parametersmentioning

confidence: 84%

Section: Introductionmentioning

confidence: 99%

Bayesian Inference for Generalized Linear Model with Linear Inequality Constraints

Ghosal,

Ghosh

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Meinshausen ( 2013 ) and Slawski and Hein ( 2013 ) show that as a regularizer, sign constraints can provide the same convergence rate as the lasso, providing that the design matrix satisfies the positive eigenvalue condition, and the sign of coefficients is known based on prior knowledge. Koike and Tanoue ( 2019 ) extends the result to general convex loss functions and nonlinear response variables, including logistic regression.…”

Section: Related Workmentioning

confidence: 86%

Targeting resources efficiently and justifiably by combining causal machine learning and theory

Ali

2022

Front. Artif. Intell.

View full text Add to dashboard Cite

IntroductionEfficient allocation of limited resources relies on accurate estimates of potential incremental benefits for each candidate. These heterogeneous treatment effects (HTE) can be estimated with properly specified theory-driven models and observational data that contain all confounders. Using causal machine learning to estimate HTE from big data offers higher benefits with limited resources by identifying additional heterogeneity dimensions and fitting arbitrary functional forms and interactions, but decisions based on black-box models are not justifiable.MethodsOur solution is designed to increase resource allocation efficiency, enhance the understanding of the treatment effects, and increase the acceptance of the resulting decisions with a rationale that is in line with existing theory. The case study identifies the right individuals to incentivize for increasing their physical activity to maximize the population's health benefits due to reduced diabetes and heart disease prevalence. We leverage large-scale data from multi-wave nationally representative health surveys and theory from the published global meta-analysis results. We train causal machine learning ensembles, extract the heterogeneity dimensions of the treatment effect, sign, and monotonicity of its moderators with explainable AI, and incorporate them into the theory-driven model with our generalized linear model with the qualitative constraint (GLM_QC) method.ResultsThe results show that the proposed methodology improves the expected health benefits for diabetes by 11% and for heart disease by 9% compared to the traditional approach of using the model specification from the literature and estimating the model with large-scale data. Qualitative constraints not only prevent counter-intuitive effects but also improve achieved benefits by regularizing the model.

show abstract

“…The linear inequality restrictions in linear regression models have been widely investigated to find the least square estimator and its properties such as bias, the mean square error and the efficiency over inequality restrictions (see Judge and Takayama (1966), Liew (1976), Lovell and Prescott (1970), Skarpness (1987, 1986), and Ohtani (1987)). Recently, in the area of big data, it has been proved that the incorporation of non-negative restrictions provides sparsity without using any regularization in linear regression models (Meinshausen 2013, Slawski andHein 2013) and also in generalized linear models (Koike and Tanoue 2019). Obviously, ignoring this type of information can impact the model estimation procedure, and decrease the accuracy of predictions.…”

Section: Introductionmentioning

confidence: 99%

Inequality Restricted Estimator for Gamma Regression: Bayesian approach as a solution to the Multicollinearity

Seifollahi¹,

Bevrani²,

Kamary³

2023

Preprint

View full text Add to dashboard Cite

In this paper, we consider the multicollinearity problem in the gamma regression model when model parameters are linearly restricted. The linear restrictions are available from prior information to ensure the validity of scientific theories or structural consistency based on physical phenomena. In order to make relevant statistical inference for a model any available knowledge and prior information on the model parameters should be taken into account. This paper proposes therefore an algorithm to acquire Bayesian estimator for the parameters of a gamma regression model subjected to some linear inequality restrictions. We then show that the proposed estimator outperforms the ordinary estimators such as the maximum likelihood and ridge estimators in term of pertinence and accuracy through Monte Carlo simulations and application to a real dataset.

show abstract

Oracle inequalities for sign constrained generalized linear models

Cited by 6 publications

References 32 publications

Bayesian Inference for Generalized Linear Model with Linear Inequality Constraints

Bayesian Inference for Generalized Linear Model with Linear Inequality Constraints

Targeting resources efficiently and justifiably by combining causal machine learning and theory

Inequality Restricted Estimator for Gamma Regression: Bayesian approach as a solution to the Multicollinearity

Contact Info

Product

Resources

About