Are pilot trials useful for predicting randomisation and attrition rates in definitive studies: A review of publicly funded trials

Abstract

Background/aims:External pilot trials are recommended for testing the feasibility of main or confirmatory trials. However, there is little evidence that progress in external pilot trials actually predicts randomisation and attrition rates in the main trial. To assess the use of external pilot trials in trial design, we compared randomisation and attrition rates in publicly funded randomised controlled trials with rates in their pilots.Methods:Randomised controlled trials for which there was an external pilot trial were identified from reports published between 2004 and 2013 in the Health Technology Assessment Journal. Data were extracted from published papers, protocols and reports. Bland–Altman plots and descriptive statistics were used to investigate the agreement of randomisation and attrition rates between the full and external pilot trials.Results:Of 561 reports, 41 were randomised controlled trials with pilot trials and 16 met criteria for a pilot trial with sufficient data. Mean attrition and randomisation rates were 21.1% and 50.4%, respectively, in the pilot trials and 16.8% and 65.2% in the main. There was minimal bias in the pilot trial when predicting the main trial attrition and randomisation rate. However, the variation was large: the mean difference in the attrition rate between the pilot and main trial was −4.4% with limits of agreement of −37.1% to 28.2%. Limits of agreement for randomisation rates were −47.8% to 77.5%.Conclusion:Results from external pilot trials to estimate randomisation and attrition rates should be used with caution as comparison of the difference in the rates between pilots and their associated full trial demonstrates high variability. We suggest using internal pilot trials wherever appropriate.

The association of working alliance, outcome expectation, adherence and self‐efficacy with clinical outcomes for Achilles tendinopathy: A feasibility cohort study (the MAP study)

“…Of these participants 66% were retained and completed all three questionnaires. While the difference in the attrition rates between feasibility studies and their associated full trial demonstrates high variability (Cooper, Whitehead, Pottrill, Julious, & Walters, ), strategies to maximize retention were reported in the accompanying process evaluation (Mallows et al, ). Only three people started but did not complete the initial questionnaire resulting in a 95% conversion rate.…”

Role of feasibility and pilot studies in randomised controlled trials: a cross-sectional study

“…However, how they are reported, and in what context, requires caution especially when interpreting the findings and extrapolating these to the delivery of a definitive trial. 3 11 12 17 …”

“… 1 Research by Cooper et al has also shown variability between external pilots and the prediction for randomisation and attrition rates. 3 As a result, much attention has primarily focused on the design, conduct and analysis of clinical research to determine where improvements are needed to reduce waste in research.…”

“…More recently, research has begun to explore the differences between internal and external pilot studies and their contribution to main trials, and the appropriateness of pilot and feasibility studies for estimating the sample size. 3 11 12 …”

<p>Lifestyle Matters Randomized Controlled Trial of a Preventive Health Intervention for Older People: Qualitative Sub Study with Participants and Intervention Facilitators</p>

“…28 The Lifestyle Matters trial 15 highlighted the difficulties of recruiting those most in need of such an intervention; namely people who had become lonely, isolated, and inactive. The previously undertaken feasibility study of the same intervention 6 recruited those who met these criteria, demonstrating the limits of using recruitment data from feasibility studies to predict recruitment in larger randomized trials where a rapid recruitment drive is usually required, 29 and in particular recruitment to trials of interventions with a group element such as Lifestyle Matters.…”