A Bayesian approach for sample size determination in method comparison studies

Yin, Kunshan; Choudhary, Pankaj; Varghese, Diana Grace; Goodman, Steven R.

doi:10.1002/sim.3124

Cited by 14 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There exist different methods to calculate the sample size necessary to achieve the actual power for normal data, while accounting for sampling variability in the parameters relevant for sample size estimation, such as the sample variance. These methods may involve an adjustment of the planned power [1,2], an adjustment of the sample standard deviation (SD) [3], or may utilize a Bayesian approach using prior information from the existing data [4][5][6][7].…”

Section: Responsementioning

confidence: 99%

Response

Chen

Zhang

et al. 2013

Clinical Trials

View full text Add to dashboard Cite

We thank Dr Fay for his letter, in which he suggests an alternative approach for evaluating sample size using the existing data [1]. There exist different methods to calculate the sample size necessary to achieve the actual power for normal data, while accounting for sampling variability in the parameters relevant for sample size estimation, such as the sample variance. These methods may involve an adjustment of the planned power [1,2], an adjustment of the sample standard deviation (SD) [3], or may utilize a Bayesian approach using prior information from the existing data [4][5][6][7].In our article [3], we first show that using the sample SD or average SD from existing data as an estimate of the true (population) SD can lead to underpowered clinical trials. Next, we suggest a variety of methods (Table 7) for choosing the appropriate SD, which can in turn be used in existing, standard sample size formulas. We proceed by suggesting guidelines for choosing the SD that results in the actual power being equal to, or exceeding the planned power, at least 80% of the time. Our recommendations are based on the results of simulations that utilize scenarios in which different numbers of preliminary studies are available. Dr Fay et al. suggest a different approach. First, a function, that averages the 'fixed-n' power functions over the assumed distribution of the SDs, is defined as the 'over-all' power function. Next, the 'over-all' power function is set to the desired 'actual' power, and the Beta (planned power) in the 'fixed-n' functions is adjusted such that the 'over-all' power function is equal to the desired actual power [8]. In his letter [1], Dr Fay extends this method to the case where multiple preliminary studies are available, by substituting a weighted variance estimate into the 'fixed-n' power functions, averaging them over the distribution of the weighted estimate, and then adjusting the planned power parameter accordingly.It is worth noting that our study only focused on the variability in the sample variance estimates, but not on the variability in the effect size derived from pilot studies and expert recommendations. Since sample effect size is a random variable, the effect size derived from a single pilot study is likely to underestimate or overestimate the population effect size. Furthermore, since effect size often depends on the population variance (e.g., Cohen's d and the confidence interval for the population value of Cohen's d [9,10]), expert opinion regarding the difference in the outcome measures across two groups is also affected by the SD estimate. Therefore, research advances in the direction of addressing these two sources of uncertainty may result in a smaller number of underpowered clinical trials.In sum, it appears that both approaches [1,3] can be successful in adjusting for the bias resulted from the uncertainty inherent in the population variance estimate used in sample size calculations. The method described in Fay et al.[8] achieves this by adjusting the planned power parameter (Beta) ...

show abstract

Section: Responsementioning

confidence: 99%

Response

Chen

Zhang

et al. 2013

Clinical Trials

View full text Add to dashboard Cite

show abstract

“…Determining the target sample size is an important step in any study design and should be considered and justified a priori. However, in the design of agreement studies, sample size determination often does not receive the same level of attention as the choice of method for assessing agreement [ 4 , 5 ].…”

Section: Introductionmentioning

confidence: 99%

A descriptive study of samples sizes used in agreement studies published in the PubMed repository

Han

Tan

Julious

et al. 2022

BMC Med Res Methodol

View full text Add to dashboard Cite

Introduction A sample size justification is required for all studies and should give the minimum number of subjects to be recruited for the study to achieve its primary objective. The aim of this review is to describe sample sizes from agreement studies with continuous or categorical endpoints and different methods of assessing agreement, and to determine whether sample size justification was provided. Methods Data were gathered from the PubMed repository with a time interval of 28th September 2018 to 28th September 2020. The search returned 5257 studies of which 82 studies were eligible for final assessment after duplicates and ineligible studies were excluded. Results We observed a wide range of sample sizes. Forty-six studies (56%) used a continuous outcome measure, 28 (34%) used categorical and eight (10%) used both. Median sample sizes were 50 (IQR 25 to 100) for continuous endpoints and 119 (IQR 50 to 271) for categorical endpoints. Bland–Altman limits of agreement (median sample size 65; IQR 35 to 124) were the most common method of statistical analysis for continuous variables and Kappa coefficients for categorical variables (median sample size 71; IQR 50 to 233). Of the 82 studies assessed, only 27 (33%) gave justification for their sample size. Conclusions Despite the importance of a sample size justification, we found that two-thirds of agreement studies did not provide one. We recommend that all agreement studies provide rationale for their sample size even if they do not include a formal sample size calculation.

show abstract

“…However, the accuracy of the model is an issue of this method (Bilić-Zulle, 2011). The method comparison models in previous studies (Yin et al, 2008;Choudhary and Kunshan, 2010) are working well with moderate or large sample sizes. However, a method comparison model with small sample sizes is vital for clinical trials.…”

Section: Introductionmentioning

confidence: 99%

Bayesian linear regression model for method comparison studies

2022

View full text Add to dashboard Cite

Highlights• The proposed Bayesian linear regression model has higher accuracy than other existing models.• It requires less time for model fitting and performed well, specifically with small sample sizes.• The proposed model is used to develop a methodology for agreement evaluation between two methods.• This methodology can be used for both balanced and unbalanced method comparison data designs.

show abstract

A Bayesian approach for sample size determination in method comparison studies

Cited by 14 publications

References 23 publications

Response

Response

A descriptive study of samples sizes used in agreement studies published in the PubMed repository

Bayesian linear regression model for method comparison studies

Contact Info

Product

Resources

About