A reliability generalization of 51 samples employing one of the NEO personality scales was conducted. Reliability generalization is a meta-analytic method for examining the variability in the reliability of scores by determining which sample characteristics are related to differences in score reliability. It was found that there was a large amount of variability in the reliability of NEO scores, both between and within personality domains. The sample characteristics that are related to score reliability were dependent on NEO domain. Agreeableness scores appear to be the weakest of the domains assessed by the NEO scales in terms of reliability, particularly in clinical samples, for male-only samples, and when temporal consistency was the criterion for reliability. The reliability of Openness to Experience scores was low when the NEO-Five Factor Inventory was used. The advantages of conceptualizing reliability as a property of scores, and not tests, are discussed.
Several methods of constructing confidence intervals (CIs) for Spearman's rho were tested in a Monte Carlo investigation. A total of 2,000 samples of sizes 10, 50, and 200 were randomly drawn from bivariate normal populations with p, equal to .00, .29, .43, .58, .73, and .89. Each method for computing a 95% CI around p3 was evaluated with regard to size in the null case and power and coverage in non-null cases. Fisher's z transformation of r, worked well provided N was not small and Ps was not too large. The CIs constructed using the variance estimate for product-moment correlations had coverages that were consistently too liberal. Kraemer's method for establishing CIs produced coverages that were conservative. An empirical attempt to adjust the Fisher CI maintained Type I error rate near the nominal level in all cases with no loss of power. Arguments are made for the continued use of r, in behavioral research.
Reliability generalization (RG) is a measurement meta-analytic method used to explore the variability in score reliability estimates and to characterize the possible sources of this variance. This article briefly summarizes some RG considerations. Included is a description of how reliability confidence intervals might be portrayed graphically. The article includes tabulations across various RG studies, including how frequently authors (a) report score reliabilities for their own data, (b) conduct reliability induction, or (c) do not even mention reliability.Reliability generalization (RG), a meta-analytic method used to explore variability in reliability estimates and characterize the sources of this variance, may be something like a safety pin-simple, yet quite useful. Although the analogy may be debated, there is little question that an increasing number of RG studies have been published, with researchers using this meta-analytic method to evaluate the sources of measurement error variance as they occur for scores across studies using a given instrument or sets of instruments measuring a given construct. Described as "a potentially powerful method with which to characterize and explore variance in score reliability" (Vacha-
The CAGE is a commonly used alcohol screening instrument. Although considerable work has been done on the validity of CAGE scores, relatively little information is available on their reliability. Reliability induction and generalization studies were performed for the CAGE. Of the 259 studies available for analysis, only 19 (7.3%) contained reliability information for the sample scores. Thirteen (5.0%) and 227 (87.6%) articles made what are designated as reliability induction by report and reliability induction by omission. The median internal consistency reliability across 22 samples was .74, with a range of .52 to .90. Sample age was the only identified sample characteristic that demonstrated a statistically significant relationship with CAGE score reliability.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.