How Appropriate are Popular Sample Size Formulas?

Kupper, Lawrence L.; Hafner, Kerry B.

doi:10.1080/00031305.1989.10475628

Cited by 116 publications

(103 citation statements)

References 6 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…8 severely overestimates the attained tolerance probability, and thus it underestimates the necessary sample size to meet the selected criterion. Similar findings were reported in Kupper and Hafner (1989) for interval estimation of one-and two-sample problems. In the present case, the parameter value of N À 4 ð Þ 1 À r 2 XZ À Á s 2 XZ is used in place of SSE XZ , so the variability in SSE XZ has been neglected in sample size calculations.…”

Section: Simulation Studysupporting

confidence: 77%

“…Note that these two concerns of expected magnitude and tolerance probability of half-width have been discussed in Kupper and Hafner (1989) for simpler situations of one-and two-sample problems and in Kelley and Maxwell (2008) for multiple linear regression. Although the notion of expected width is widely covered in standard texts for sample size determination, the assurance of tolerance probability approach is recommended by Kupper and Hafner.…”

Section: Confidence Intervals Of Interaction Effectsmentioning

confidence: 99%

See 1 more Smart Citation

The impact of ignoring random features of predictor and moderator variables on sample size for precise interval estimation of interaction effects

Shieh

2011

Behav Res

View full text Add to dashboard Cite

The influence of the joint distribution of predictor and moderator variables on the identification of interactions has been well described, but the impact on sample size determinations has received rather limited attention within the framework of moderated multiple regression (MMR). This article investigates the deficiency in sample size determinations for precise interval estimation of interaction effects that can result from ignoring the stochastic nature of continuous predictor and moderator variables in MMR. The primary finding of our examinations is that failure to accommodate the distributional properties of regressors can lead to underestimation of the necessary sample size and distortion of the desired interval precision. In order to take account of the randomness of regressor variables, two general and effective procedures for computing sample size estimates are presented. Moreover, corresponding programs are provided to facilitate use of the suggested approaches. This exposition helps to correct drawbacks in the existing techniques and to advance the practice of reporting confidence intervals in MMR analyses.Keywords Moderation . Precision . Sample size Moderated multiple regression (MMR) has been extensively employed to study the interaction effects between predictor and moderator variables in management, psychology, education, and related disciplines. It follows from the comprehensive reviews of Stone-Romero, Alliger, and Aguinis (1994), Aguinis (1995), Aguinis and Stone-Romero (1997), Aguinis, Beaty, Boik, and Pierce (2005), and relatedwork that most of the methodological research in MMR has been concerned with the statistical power of hypothesis testing for detecting moderating effects. Although null hypothesis significance testing is useful in various applications, the dichotomous accept-reject decision ignores other useful information in its analysis. As an alternative, the notion of interval estimation has been stressed in studies such as Hahn and Meeker (1991), Steiger andFouladi (1997), andSmithson (2003). Accordingly, the inferential procedures of interval estimators are strongly recommended by Wilkinson and the American Psychological Association Task Force on Statistical Inference (1999), as well as the Publication Manualof the American Psychological Association (American Psychological Association, 2001). Since confidence intervals constructed with the desired reliability are more informative about the location of a targeted parameter, they should be the best reporting strategy in practical study. However, the methodological artifacts and statistical implications associated with interval estimation of moderating effects have received little attention within the framework of MMR.The interactional formulation of MMR can be viewed as a special case of the statistical linear models, and so the inferential procedures of hypothesis testing and interval estimation of moderation can be conducted with standard methods and software packages for linear regression analysis. In this article, we ...

show abstract

Section: Simulation Studysupporting

confidence: 77%

Section: Confidence Intervals Of Interaction Effectsmentioning

confidence: 99%

The impact of ignoring random features of predictor and moderator variables on sample size for precise interval estimation of interaction effects

Shieh

2011

Behav Res

View full text Add to dashboard Cite

show abstract

“…But it is evident from the reported results in these tables that the sample sizes increase with an increasing value of δ* when all other factors are fixed. This particular phenomenon differs from the common sample size procedures for precise interval estimation of mean differences, in which the required sample sizes do not vary with the magnitude of population mean difference, as shown in Kupper and Hafner (1989) and Wang and Kupper (1997).…”

Section: Numerical Illustration Of Sample Size Calculationsmentioning

confidence: 89%

“…Hence, the sample sizes computed by the expected width approach tend to be inadequate to guarantee the desired assurance level of interval width. Consequently, Kupper and Hafner (1989) recommended the assurance probability approach over the expected width criterion for sample size determination, although the notion of expected width is widely covered in standard texts for sample size determination of precise interval estimation. It is noteworthy that the expected width and assurance probability principles are closely related to the two distinct principles of unbiasedness and consistency in statistical point estimation.…”

Section: Numerical Illustration Of Sample Size Calculationsmentioning

confidence: 99%

Confidence intervals and sample size calculations for the standardized mean difference effect size between two normal populations under heteroscedasticity

Shieh

2013

Behav Res

View full text Add to dashboard Cite

The use of effect sizes and associated confidence intervals in all empirical research has been strongly emphasized by journal publication guidelines. To help advance theory and practice in the social sciences, this article describes an improved procedure for constructing confidence intervals of the standardized mean difference effect size between two independent normal populations with unknown and possibly unequal variances. The presented approach has advantages over the existing formula in both theoretical justification and computational simplicity. In addition, simulation results show that the suggested one-and two-sided confidence intervals are more accurate in achieving the nominal coverage probability. The proposed estimation method provides a feasible alternative to the most commonly used measure of Cohen's d and the corresponding interval procedure when the assumption of homogeneous variances is not tenable. To further improve the potential applicability of the suggested methodology, the sample size procedures for precise interval estimation of the standardized mean difference are also delineated. The desired precision of a confidence interval is assessed with respect to the control of expected width and to the assurance probability of interval width within a designated value. Supplementary computer programs are developed to aid in the usefulness and implementation of the introduced techniques. Kirk (1996), Kline (2004), Olejnik and Algina (2000), Richardson (1996), Rosenthal, Rosnow, and Rubin (2000), Rosenthal (2003), andVacha-Haase andThompson (2004). It has steadily become a general consensus in the methodological literature of behavior, education, management, and related disciplines that effect sizes accompanied by their corresponding confidence intervals are perhaps the best approach for conveying quantitative information in applied research.According to the general review of Ferguson (2009), effect sizes can be categorized into four general classes: (1) group difference, (2) strength of association, (3) corrected estimates, and (4) risk estimates. The group difference indices estimate the magnitude of difference between two or more groups, and Cohen's d (Cohen, 1969) is the most commonly used measure across virtually all disciplines of the social sciences. Specifically, Cohen's d is an estimate of the standardized mean difference that reflects the difference between two sample means divided by their pooled sample standard deviation under homoscedasticity. For the purpose of measuring the size of effect between two treatment groups with unequal variances, Cohen's d is no longer a proper estimator, because its standardizer, the Electronic supplementary material The online version of this article

show abstract

“…Greenland (1988) introduces the concept of 'discriminatory power' for calculating sample size based on confidence interval and Daly (1991) proposes to translate the standard methods for calculating sample size based on power into the confidence interval framework. Kupper and Hafner (1989) for an evaluation of the performance of sample size calculations based on confidence intervals. Finally, many authors agree that when the main aim of the study is estimation the average length of the confidence interval can be used as an criterion for sample size calculations (Beal, 1989;Daly, 1991;Greenland, 1988;Maxwell and Kelley, 2007).…”

Section: Sample Size Calculations Based On Icc(21)mentioning

confidence: 99%

Design Based on Intra-Class Correlation Coefficients

Lew¹,

Doros²

2010

American Journal of Biostatistics

View full text Add to dashboard Cite

Problem statement: Reliability studies are concerned with the study of "consistency" or "repeatability" of measurements. Often times (but not always) the reliability coefficients are Intra-class Correlation Coefficients (ICC). Depending on the design or the conceptual intent of the study there are three types of intra-class correlation coefficients, termed intra-class correlation coefficients Case 1, 2 and 3, for measuring the reliability of a single interval measure. While methods for sample size calculations for intra-class correlation coefficients in Case 1 are available and implemented in Power Analysis and Sample Size System (PASS); to our knowledge, no methods based on intra-class correlation coefficients in Case 2 and 3 are available. Objective: Develop a method for calculating the size of a reliability study based on intra-class correlation coefficients Case 1 and 2. Approach: A practical method for computing sample size using simulations was proposed. We proposed to compute sample size based on the expected width of the confidence interval. For a given target value of the intra-class correlation coefficient, the proposed method chooses the design assures a 95% confidence interval with average length shorter than a pre-specified value. The applicability of the proposed method in practice for intra-class coefficients Case 2 was supported by demonstrating three invariance properties of the proposed confidence intervals. Results: Tables with sample size requirements were derived and displayed. A program for carrying out the calculations was developed in R. The method was used to size a trial aimed to study the reliability of a scale that measures the cleanness of the colon at the time of colonoscopy. Conclusion: A simple method for sample size calculation for intra-class correlation coefficient Case 1 and 2 based on the average length of confidence intervals was proposed. The proposed was implemented by the authors in R (freely available software). Three invariance properties of the confidence intervals for the intra-class correlation coefficients Case 2 were studied by simulations. These properties are an important tool when considering the design of this type of studies.

show abstract

How Appropriate are Popular Sample Size Formulas?

Cited by 116 publications

References 6 publications

The impact of ignoring random features of predictor and moderator variables on sample size for precise interval estimation of interaction effects

The impact of ignoring random features of predictor and moderator variables on sample size for precise interval estimation of interaction effects

Confidence intervals and sample size calculations for the standardized mean difference effect size between two normal populations under heteroscedasticity

Design Based on Intra-Class Correlation Coefficients

Contact Info

Product

Resources

About