Multilevel Linear Models, Gibbs Samplers and Multigrid Decompositions (with Discussion)

Zanella, Giacomo; Roberts, Gareth O.

doi:10.1214/20-ba1242

Cited by 11 publications

(7 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These first of these is implementing constraints on the

and

parameters to improve model convergence. While this may seem counterintuitive, such identifiability constraints have been shown to dramatically improve convergence of Gibbs samplers for Gaussian mixture models ( Zanella and Roberts 2021 ). The other changes included updating

and

with fixed calculations and editing the underlying code to improve computational efficiency.…”

Section: Methodsmentioning

confidence: 99%

MetaNorm: incorporating meta-analytic priors into normalization of NanoString nCounter data

Barth,

Yang,

Xiao

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

Motivation Non-informative or diffuse prior distributions are widely employed in Bayesian data analysis to maintain objectivity. However, when meaningful prior information exists and can be identified, using an informative prior distribution to accurately reflect current knowledge may lead to superior outcomes and great efficiency. Results We propose MetaNorm, a Bayesian algorithm for normalizing NanoString nCounter gene expression data. MetaNorm is based on RCRnorm, a powerful method designed under an integrated series of hierarchical models that allow various sources of error to be explained by different types of probes in the nCounter system. However, a lack of accurate prior information, weak computational efficiency, and instability of estimates that sometimes occur weakens the approach despite its impressive performance. MetaNorm employs priors carefully constructed from a rigorous meta-analysis to leverage information from large public data. Combined with additional algorithmic enhancements, MetaNorm improves RCRnorm by yielding more stable estimation of normalized values, better convergence diagnostics and superior computational efficiency. Availability and Implementation R Code for replicating the meta-analysis and the normalization function can be found at github.com/jbarth216/MetaNorm.

show abstract

“…These first of these is implementing constraints on the

and

with fixed calculations and editing the underlying code to improve computational efficiency.…”

Section: Methodsmentioning

confidence: 99%

MetaNorm: incorporating meta-analytic priors into normalization of NanoString nCounter data

Barth,

Yang,

Xiao

et al. 2024

Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Posterior approximation via gradient-based MCMC methods encounter issues when sampling from non-differentiable distributions, such as the double expo-nential (Brooks et al, 2011). Additionally, it is straighforward to reparametrize the posterior distribution to improve the rates of convergence of either Gibbs or Hamiltonian Monte Carlo based MCMC samplers in the context of conditionally normal multilevel models (Bürkner, 2017;Zanella and Roberts, 2021).…”

Section: Derivation Of the R2-d2-m2 Priormentioning

confidence: 99%

Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior

Aguilar¹,

Bürkner²

2022

Preprint

View full text Add to dashboard Cite

The training of high-dimensional regression models on comparably sparse data is an important yet complicated topic, especially when there are many more model parameters than observations in the data. From a Bayesian perspective, inference in such cases can be achieved with the help of shrinkage prior distributions, at least for generalized linear models. However, real-world data usually possess multilevel structures, such as repeated measurements or natural groupings of individuals, which existing shrinkage priors are not built to deal with.We generalize and extend one of these priors, the R2-D2 prior by Zhang et al. (2020), to linear multilevel models leading to what we call the R2-D2-M2 prior. The proposed prior enables both local and global shrinkage of the model parameters. It comes with interpretable hyperparameters, which we show to be intrinsically related to vital properties of the prior, such as rates of concentration around the origin, tail behavior, and amount of shrinkage the prior exerts.We offer guidelines on how to select the prior's hyperparameters by deriving shrinkage factors and measuring the effective number of non-zero model coefficients. Hence, the user can readily evaluate and interpret the amount of shrinkage implied by a specific choice of hyperparameters.Finally, we perform extensive experiments on simulated and real data, showing that our prior is well calibrated, has desirable global and local regularization properties and enables the reliable and interpretable estimation of much more complex Bayesian multilevel models than was previously possible.

show abstract

“…Nevertheless, from the point of view of this article it is worth considering their impact on algorithmic performance. Theorem 6 of Zanella and Roberts (2020) derives the relaxation times of the vanilla Gibbs sampler for crossed effect models with Gaussian likelihood. Some constraints can make the vanilla Gibbs sampler scalable, whereas others cannot.…”

Section: Related Literature and Alternative Approachesmentioning

confidence: 99%

“…Both such extensions are non-trivial and would lead to a significant generalization of the result. In order to consider unbalanced designs, one needs to extend proof techniques based on multigrid decompositions (Zanella and Roberts, 2020;Papaspiliopoulos et al, 2020), which exploit the exact independence between appropriate reparametrizations of θ, to cases of weak dependence. In this direction, the proof strategy of Ghosh et al (2020) can be interesting, which exploits appropriate concentration inequalities to provide upper bounds on the convergence rate of coordinate-wise optimization for computing the MAP estimator for crossed effects with unbalanced levels.…”

Section: Random Graphs Weak Dependence and Complexitymentioning

confidence: 99%

Scalable computation for Bayesian hierarchical models

Papaspiliopoulos¹,

Stumpf-Fétizon²,

Zanella³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The article is about algorithms for learning Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. It focuses on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences, and illustrates the methodology on two challenging real data analyses on predicting electoral results and real estate prices respectively. The posterior dependence in both classes is sparse: in crossed random effects models it resembles a random graph, whereas in nested multilevel models it is tree-structured. For each class we develop a framework for scalable computation based on collapsed Gibbs sampling and belief propagation respectively. We provide a number of negative (for crossed) and positive (for nested) results for the scalability (or lack thereof) of methods based on sparse linear algebra, which are relevant also to Laplace approximation methods for such models. Our numerical experiments compare with off-the-shelf variational approximations and Hamiltonian Monte Carlo. Our theoretical results, although partial, are useful in suggesting interesting methodologies and lead to conclusions that our numerics suggest to hold well beyond the scope of the underlying assumptions.

show abstract

Multilevel Linear Models, Gibbs Samplers and Multigrid Decompositions (with Discussion)

Cited by 11 publications

References 40 publications

MetaNorm: incorporating meta-analytic priors into normalization of NanoString nCounter data

MetaNorm: incorporating meta-analytic priors into normalization of NanoString nCounter data

Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior

Scalable computation for Bayesian hierarchical models

Contact Info

Product

Resources

About