Bayesian meta‐analysis of coefficient alpha

Abstract

The current paper describes and illustrates a Bayesian approach to the meta-analysis of coefficient alpha. Alpha is the most commonly used estimate of the reliability or consistency (freedom from measurement error) for educational and psychological measures. The conventional approach to meta-analysis uses inverse variance weights to combine information from independent studies to provide an overall estimate. The Bayesian approach provides similar estimates to the conventional approach if a diffuse prior is used. However, the Bayesian approach also provides 'shrunken' local estimates of reliability in each context. The amount of shrinkage depends upon both the variability in the underlying populations and the sampling variance of the local estimates. Advantages of the approach are the estimation of individual studies adjusted for sampling error and the application of meta-analytic results to new local studies in which the local study 'borrows strength' from the meta-analysis. The ability to borrow strength for the new local studies is particularly useful in applied work in which the estimate of the local parameter is of primary interest. The approach is illustrated by the analysis of studies of the reliability of the General Ethnicity Questionnaire - Abridged, a measure of identification with the culture of one's heritage and the culture of one's host country. Copyright © 2013 John Wiley & Sons, Ltd.

Bayesian meta‐analysis of Cronbach's coefficient alpha to evaluate informative hypotheses

“…We agree with Brannick and Zhang () on the above points. Meanwhile, an additional and perhaps more important benefit of adopting a Bayesian approach is formal statistical model evaluation and model selection.…”

“…In our results, prior specification on had little effect on the posterior inference. This could be because our meta‐analysis data‐generating model (which is the same as Rodriguez and Maeda () and Brannick and Zhang ()'s model) has a hierarchical structure with variance v i given from the data. Still, as we showed in Section 4.1, the resultant Bayes factor can be affected by prior specification of because it affects the relative complexity c m of the informative hypothesis.…”

“…Thus far, our model specifications are the same as those in previous studies. Our data generating model (likelihood) under H u (Eqn ) is the same as Rodriguez and Maeda () and Brannick and Zhang (), and the prior on μ i (Eqn ) is the same as Brannick and Zhang (). However, henceforth, we use different prior specifications for the priors of and τ 2 for the following reasons.…”

“…First, Brannick and Zhang () set a normal prior on . However, because our primary focus of interest is on rather than , we set priors on instead of .…”

“…Recently, Brannick and Zhang () proposed a new Bayesian approach for the meta‐analysis of coefficient alpha. Using the underlying model proposed by Rodriguez and Maeda (); Brannick and Zhang () replaced the classical frequentist inference with a Bayesian inference using Markov chain Monte Carlo (MCMC) methods. They describe three benefits of the Bayesian approach.…”