The cohort multiple randomized controlled trial design was found to be highly susceptible to low statistical power and internal validity biases

Abstract

ObjectivesThe “cohort multiple randomized controlled trial” (cmRCT) is a recent innovation by which novel interventions are trialed within large longitudinal cohorts of patients to gain efficiencies and align trials more closely to standard clinical practice. The use of cmRCTs is outpacing its methodological understanding, and more appropriate methods for designing and analyzing such trials are urgently needed.Study Design and SettingWe established the UK Comprehensive Longitudinal Assessment of Salford Integrated Care cohort of 4,377 patients with long-term conditions within which we are conducting a cmRCT (“Proactive Telephone Coaching and Tailored Support”) of telephone-based health coaching.ResultsWe identify some key methodological challenges to the use of the cmRCT in actual practice. Principal are issues around statistical power, sample size, and treatment effect estimation, for which we provide appropriate methods. Sampling procedures commonly applied in conventional RCTs can result in unintentional selection bias. The fixed data collection points that feature in cmRCTs can also threaten validity.ConclusionThe cmRCT may offer advantages over conventional trial designs. However, a cmRCT requires appropriate power calculation, sampling, and analysis procedures; else, studies may be underpowered or subject to validity biases. We offer solutions to some of the key issues, but further methodological investigations are needed. Cohort multiple RCT–specific Consolidated Standards of Reporting Trials guidance may be indicated.

The Trials within Cohorts design faced methodological advantages and disadvantages in the exercise oncology setting

“…During the recruitment phase, the actual acceptance rate turned out to be lower than expected, and we had to increase our sample size and extend our recruitment period [11,32]. Fortunately, UMBRELLA Fit was performed within a dynamic cohort with continuous recruitment of eligible patients.…”

“…Fortunately, UMBRELLA Fit was performed within a dynamic cohort with continuous recruitment of eligible patients. For trials performed in fixed cohorts, updating the sample size could be problematic, resulting in underpowered trials [11]. Because of intervention refusal, the final sample size of our TwiCs study was larger compared with the sample sizes of the conventional RCTs.…”

“…In future trials using the TwiCs design, we recommend performing a pilot study first, especially in fixed cohorts, to get more insight into the acceptance rate of the intervention and recruitment timelines. This was also recommended by Reeves et al [11] who performed a trial within a fixed cohort, which makes pilot testing even more important. When noncompliance is higher than expected, resulting in a larger sample size, the trial might not be feasible in the cohort when the fixed cohort contains not enough eligible participants.…”

“…This prevents contamination and reduces dropout of controls. At the same time, effect sizes are affected by the percentage of patients agreeing to undergo the intervention [11].…”

Advance care planning for vulnerable older adults within an Accountable Care Organization: study protocol for the IMPACT randomised controlled trial

“…The primary nuance for estimating statistical power is the use of Zelen’s prerandomisation design, whereby only patients randomised to the nurse navigator group will be approached for consent. This naturally attenuates any presumed effect of the intervention, as a proportion of patients will not receive the intervention 62. Based on a previous randomised trial of ACP strategies conducted within the Veterans Affairs system, we assumed that 44% of patients randomised to the nurse navigator group will consent to participate 63.…”