Going Beyond Convergence in Bayesian Estimation: Why Precision Matters Too and How to Assess It

Zitzmann, Steffen; Hecht, Martin

doi:10.1080/10705511.2018.1545232

Cited by 74 publications

(76 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, all ESS values exceeded 1,800 in our investigation. This amount exceeds the recommendation by Zitzmann and Hecht (2019) . Therefore, we believe that the chains in the simulation represent adequate precision.…”

mentioning

confidence: 63%

“…Another index that can be checked is the effective sample size (ESS), which is directly linked to the degree of dependency (or autocorrelation ) within the chain. Zitzmann and Hecht (2019) recommend that ESSs over 1,000 are required to ensure that there is enough precision in the chain. Simulation results indicated that, although the post burn-in portions of the chain were only 2,500 iterations, all of the parameters exceeded the minimum of ESS = 1,000 in the cells examined.…”

Section: Proof Of Concept Simulation: Illustrating the Impact Of Priomentioning

confidence: 99%

See 1 more Smart Citation

The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations Using an Interactive Shiny App

2020

View full text Add to dashboard Cite

The current paper highlights a new, interactive Shiny App that can be used to aid in understanding and teaching the important task of conducting a prior sensitivity analysis when implementing Bayesian estimation methods. In this paper, we discuss the importance of examining prior distributions through a sensitivity analysis. We argue that conducting a prior sensitivity analysis is equally important when so-called diffuse priors are implemented as it is with subjective priors. As a proof of concept, we conducted a small simulation study, which illustrates the impact of priors on final model estimates. The findings from the simulation study highlight the importance of conducting a sensitivity analysis of priors. This concept is further extended through an interactive Shiny App that we developed. The Shiny App allows users to explore the impact of various forms of priors using empirical data. We introduce this Shiny App and thoroughly detail an example using a simple multiple regression model that users at all levels can understand. In this paper, we highlight how to determine the different settings for a prior sensitivity analysis, how to visually and statistically compare results obtained in the sensitivity analysis, and how to display findings and write up disparate results obtained across the sensitivity analysis. The goal is that novice users can follow the process outlined here and work within the interactive Shiny App to gain a deeper understanding of the role of prior distributions and the importance of a sensitivity analysis when implementing Bayesian methods. The intended audience is broad (e.g., undergraduate or graduate students, faculty, and other researchers) and can include those with limited exposure to Bayesian methods or the specific model presented here.

show abstract

mentioning

confidence: 63%

Section: Proof Of Concept Simulation: Illustrating the Impact Of Priomentioning

confidence: 99%

The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations Using an Interactive Shiny App

2020

View full text Add to dashboard Cite

show abstract

“…The efficiency of the estimation procedure was evaluated by considering ESS (Kass, Carlin, Gelman, & Neal, 1998), indicating the degree of precision with which the empirical mean of the MCMC chains approximates the expected value of the posterior distribution (Lüdtke, Robitzsch, & Wagner, 2018). Following Zitzmann and Hecht (2019), we considered an ESS above 400 for all parameters as sufficient.…”

Section: Resultsmentioning

confidence: 99%

“…In some cells of the simulation design with N ≤ 500, proportions of replications with ESSs for all parameters higher than 400 were somewhat lower than convergence rates as evaluated on the basis of PSRF values. This indicates that although the chains converged and mixed well, the parameter space was explored rather slowly and more iterations might be needed to ensure good approximation of the posterior mean (Zitzmann & Hecht, 2019). Further assessments of convergence behaviour of replications with PSRF values above 1.10 showed very poor, if any, mixing of the MCMC chains, with PSRF values of up to 573.45, indicating that engaged and disengaged behaviours were not separable.…”

Section: Resultsmentioning

confidence: 99%

A hierarchical latent response model for inferences about examinee engagement in terms of guessing and item‐level non‐response

Ulitzsch

Davier

Pohl

2019

Brit J Math & Statis

View full text Add to dashboard Cite

In low-stakes assessments, test performance has few or no consequences for examinees themselves, so that examinees may not be fully engaged when answering the items. Instead of engaging in solution behaviour, disengaged examinees might randomly guess or generate no response at all. When ignored, examinee disengagement poses a severe threat to the validity of results obtained from low-stakes assessments. Statistical modelling approaches in educational measurement have been proposed that account for non-response or for guessing, but do not consider both types of disengaged behaviour simultaneously. We bring together research on modelling examinee engagement and research on missing values and present a hierarchical latent response model for identifying and modelling the processes associated with examinee disengagement jointly with the processes associated with engaged responses. To that end, we employ a mixture model that identifies disengagement at the item-by-examinee level by assuming different datagenerating processes underlying item responses and omissions, respectively, as well as response times associated with engaged and disengaged behaviour. By modelling examinee engagement with a latent response framework, the model allows assessing how examinee engagement relates to ability and speed as well as to identify items that are likely to evoke disengaged test-taking behaviour. An illustration of the model by means of an application to real data is presented. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…As stopping criteria we used PSR 1:001 and ESS ! 400 for all parameters in order to ensure that the Bayesian estimates are approximated well by summary statistics for the MCMC chain (see Zitzmann & Hecht, 2019). After that, the mode of the converged chain served as the parameter estimate.…”

Section: Resultsmentioning

confidence: 99%

Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial

Hecht

Gische

Vogel

et al. 2019

Structural Equation Modeling: A Multidisciplinary Journal

Self Cite

View full text Add to dashboard Cite

Bayesian estimation has become very popular. However, run time of Bayesian models is often unsatisfactorily high. In this illustration, we show how to reduce run time by (a) integrating out nuisance model parameters and by (b) reformulating the model based on covariances and means. The core concept is to use the sample scatter matrix which is in our case Wishart distributed with the model-implied covariance matrix as the scale matrix. To illustrate this approach, we choose the popular multi-level null (intercept-only) model, provide a step-by-step instruction on how to implement this model in a multipurpose Bayesian software, and show how structural equation modeling techniques can be employed to bypass mathematically challenging derivations. A simulation study showed that run time is considerably reduced and an empirical example illustrates our approach. Further, we show how the JAGS sampling progress can be monitored and stopped automatically when convergence and precision criteria are reached.

show abstract

Going Beyond Convergence in Bayesian Estimation: Why Precision Matters Too and How to Assess It

Cited by 74 publications

References 41 publications

The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations Using an Interactive Shiny App

The Importance of Prior Sensitivity Analysis in Bayesian Statistics: Demonstrations Using an Interactive Shiny App

A hierarchical latent response model for inferences about examinee engagement in terms of guessing and item‐level non‐response

Integrating Out Nuisance Parameters for Computationally More Efficient Bayesian Estimation – An Illustration and Tutorial

Contact Info

Product

Resources

About