Sample size requirements to estimate key design parameters from external pilot randomised controlled trials: a simulation study

Abstract

BackgroundExternal pilot or feasibility studies can be used to estimate key unknown parameters to inform the design of the definitive randomised controlled trial (RCT). However, there is little consensus on how large pilot studies need to be, and some suggest inflating estimates to adjust for the lack of precision when planning the definitive RCT.MethodsWe use a simulation approach to illustrate the sampling distribution of the standard deviation for continuous outcomes and the event rate for binary outcomes. We present the impact of increasing the pilot sample size on the precision and bias of these estimates, and predicted power under three realistic scenarios. We also illustrate the consequences of using a confidence interval argument to inflate estimates so the required power is achieved with a pre-specified level of confidence. We limit our attention to external pilot and feasibility studies prior to a two-parallel-balanced-group superiority RCT.ResultsFor normally distributed outcomes, the relative gain in precision of the pooled standard deviation (SDp) is less than 10% (for each five subjects added per group) once the total sample size is 70. For true proportions between 0.1 and 0.5, we find the gain in precision for each five subjects added to the pilot sample is less than 5% once the sample size is 60. Adjusting the required sample sizes for the imprecision in the pilot study estimates can result in excessively large definitive RCTs and also requires a pilot sample size of 60 to 90 for the true effect sizes considered here.ConclusionsWe recommend that an external pilot study has at least 70 measured subjects (35 per group) when estimating the SDp for a continuous outcome. If the event rate in an intervention group needs to be estimated by the pilot then a total of 60 to 100 subjects is required. Hence if the primary outcome is binary a total of at least 120 subjects (60 in each group) may be required in the pilot trial. It is very much more efficient to use a larger pilot study, than to guard against the lack of precision by using inflated estimates.

How big should the pilot study for my cluster randomised trial be?

“…That paper reported that quite large sample sizes are needed to obtain reasonable precision and that for small sample sizes variance estimates are biased downwards. 4 Our findings also concur with previous research that indicates that the uncertainty around ICC estimates is considerable even for relatively large samples and that estimating ICCs only from pilot studies for cluster randomised trials is not optimal. 3,24 This is in spite of the fact that our brief review indicates that pilots for cluster randomised trials often include considerably more participants than pilots for individually randomised trials.…”

Acceptability and proof of concept of internet-delivered treatment for depression, anxiety, and stress in university students: protocol for an open feasibility trial

“…A sample size of 35 participants per group is the standard expectation for feasibility studies [52]. This sample size will allow us to estimate the standard deviation of the symptom outcome measures (for the proposed RCT).…”

Taxonomy for the Rehabilitation of Knee Conditions (TRAK), a Digital Intervention to Support the Self-Care Components of Anterior Cruciate Ligament Rehabilitation: Protocol of a Feasibility Study

“…There are on average 2 new patients per week in the ACL group based on local audit data, which equates to 60 patients over 30 weeks who may be eligible for recruitment. There is potential for 15 further patients to be recruited from the first phase of rehabilitation bringing the potential recruits to 75 patients [29,30]. As the inclusion criteria are broad, it is expected that most patients will be eligible, and early patient feedback indicates patients will be keen to partake.…”

“…Recruitment stops after 30 weeks, which allows the last recruited patient 7 months to participate. This is enough time to deliver on the study objectives of measuring feasibility although not necessarily enough time for patients to complete the program [30]. …”