Latent space projection predictive inference

Catalina, Alejandro; Bürkner, Paul - Christian; Vehtari, Aki

doi:10.48550/arxiv.2109.04702

Cited by 5 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Section 7 shows examples for both simpler and more complex models. For more complex models a fast approximate projection predictive approach can be used (Piironen et al, 2020;Catalina et al, 2020Catalina et al, , 2021.…”

Section: Predictive Criterion Estimationmentioning

confidence: 99%

“…1 was used by Bernardo (1999), 3 by Bernardo and Rueda (2002) and Bernardo and Juárez (2003), and 2 has not been used before, but it has some interesting frequency properties as illustrated later. 1 was also used by Goutis and Robert (1998) and Dupuis and Robert (2003), but using approximate Kullback-Leibler projections of the full model parameters to a restricted parameter space (see also later development of this approach by Piironen et al, 2020, Catalina et al, 2020, and Catalina et al, 2021.…”

Section: Intrinsic Estimationmentioning

confidence: 99%

“…Point estimates and credible regions can still be similar as illustrated in Section 7.1.4 with normal model with variance as a nuisance parameter. For further discussion and examples of defining submodels as the Kullback-Leibler projections of the full model, see papers by Piironen et al (2020), Catalina et al (2020), andCatalina et al (2021).…”

Section: Intrinsic Estimationmentioning

confidence: 99%

“…To illustrate the interesting properties, this paper focuses on simple models for which we can compute the solutions analytically or with low dimensional quadrature. The predictive criterion approach is closely connected to the projective predictive approach which replaces the exact criterion with an approximation that is computationally faster (Piironen et al, 2020;Catalina et al, 2020Catalina et al, , 2021.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

On Reparameterization Invariant Bayesian Point Estimates and Credible Regions

Vehtari

2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper considers reparameterization invariant Bayesian point estimates and credible regions of model parameters for scientific inference and communication. The effect of intrinsic loss function choice in Bayesian intrinsic estimates and regions is studied with the following findings. A particular intrinsic loss function, using Kullback-Leibler divergence from the full model to the restricted model, has strong connection to a Bayesian predictive criterion, which produces point estimates with the best predictive performance. An alternative intrinsic loss function, using Kullback-Leibler divergence from the restricted model to the full model, produces estimates with interesting frequency properties for at least some commonly used distributions, that is, unbiased minimum variance estimates of the location and scale parameters.

show abstract

Section: Predictive Criterion Estimationmentioning

confidence: 99%

Section: Intrinsic Estimationmentioning

confidence: 99%

Section: Intrinsic Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

On Reparameterization Invariant Bayesian Point Estimates and Credible Regions

Vehtari

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…So far, the implementation of the PPVS in the R (R Core Team 2023) package projpred 1 (Piironen et al 2023) has been restricted to the Gaussian, the binomial, and the Poisson response families. Recently, the latent projection (Catalina et al 2021) has extended the range of possible response families considerably, in particular to ordinal families relying on a single latent predictor (per observation), an example being the cumulative ordinal family from MASS::polr() (Venables and Ripley 2002). However, the latent projection is an approximate approach as it replaces the original projection problem with a latent projection problem.…”

Section: Introductionmentioning

confidence: 99%

Projection predictive variable selection for discrete response families with finite support

Weber,

Glass,

Vehtari

2024

Comput Stat

Self Cite

View full text Add to dashboard Cite

The projection predictive variable selection is a decision-theoretically justified Bayesian variable selection approach achieving an outstanding trade-off between predictive performance and sparsity. Its projection problem is not easy to solve in general because it is based on the Kullback–Leibler divergence from a restricted posterior predictive distribution of the so-called reference model to the parameter-conditional predictive distribution of a candidate model. Previous work showed how this projection problem can be solved for response families employed in generalized linear models and how an approximate latent-space approach can be used for many other response families. Here, we present an exact projection method for all response families with discrete and finite support, called the augmented-data projection. A simulation study for an ordinal response family shows that the proposed method performs better than or similarly to the previously proposed approximate latent-space projection. The cost of the slightly better performance of the augmented-data projection is a substantial increase in runtime. Thus, if the augmented-data projection’s runtime is too high, we recommend the latent projection in the early phase of the model-building workflow and the augmented-data projection for final results. The ordinal response family from our simulation study is supported by both projection methods, but we also include a real-world cancer subtyping example with a nominal response family, a case that is not supported by the latent projection.

show abstract

Using reference models in variable selection

et al. 2022

View full text Add to dashboard Cite

Variable selection, or more generally, model reduction is an important aspect of the statistical workflow aiming to provide insights from data. In this paper, we discuss and demonstrate the benefits of using a reference model in variable selection. A reference model acts as a noise-filter on the target variable by modeling its data generating mechanism. As a result, using the reference model predictions in the model selection procedure reduces the variability and improves stability, leading to improved model selection performance. Assuming that a Bayesian reference model describes the true distribution of future data well, the theoretically preferred usage of the reference model is to project its predictive distribution to a reduced model, leading to projection predictive variable selection approach. We analyse how much the great performance of the projection predictive variable is due to the use of reference model and show that other variable selection methods can also be greatly improved by using the reference model as target instead of the original data. In several numerical experiments, we investigate the performance of the projective prediction approach as well as alternative variable selection methods with and without reference models. Our results indicate that the use of reference models generally translates into better and more stable variable selection.

show abstract

Latent space projection predictive inference

Cited by 5 publications

References 30 publications

On Reparameterization Invariant Bayesian Point Estimates and Credible Regions

On Reparameterization Invariant Bayesian Point Estimates and Credible Regions

Projection predictive variable selection for discrete response families with finite support

Using reference models in variable selection

Contact Info

Product

Resources

About