Getting It Wrong Most of The Time: Comparing Trialists’ Choice of Primary Outcome With What Patients And Health Professionals Want

Abstract

Background: Randomised trials support improved decision-making through the data they collect. One important piece of data is the primary outcome – so called because it is what the investigators decide is the most important. Secondary outcomes provide additional information to support decision-making. We were interested in knowing how important patients and healthcare professionals consider the outcomes (especially the primary outcome) measured in a selection of published trials. Methods: The work had three stages: 1. We identified a body of late-stage trials in two clinical areas, breast cancer management and nephrology. 2. We identified the primary and secondary outcomes for these trials. 3. We randomly ordered these outcomes and presented them to patients and healthcare professionals (with experience of the clinical area), and we asked them to rank the importance of the outcomes. They were not told which outcomes trial authors considered primary and secondary. Results: In our sample of 44 trials with 46 primary outcomes, 29 patients, one patient representative and 12 healthcare professionals together ranked the primary outcome as the most important outcome 13/46 times, or 30%. Breast cancer patients and health care professionals considered the primary outcome to be the most important outcome for 8/21 primary outcomes chosen by trialists. For nephrology, the equivalent figure was 5/25. The primary outcome appeared in a respondent’s top 5 ranked outcomes 151/178 (85%) times for breast cancer and 225/259 (87%) times for nephrology even if the primary wasn’t considered the most important outcome. Conclusions: The primary outcome in a trial is the most important piece of data collected. It is used to determine how many participants are required, and it is the main piece of information used to judge whether the intervention is effective or not. Our study found that in the view of patients and healthcare professionals, teams doing trials in breast cancer management and nephrology got their choice of primary outcome wrong 70% of the time.

A good use of time? Providing evidence for how effort is invested in primary and secondary outcome data collection in trials

“…The outcomes trialists choose are not always the ones patients consider most important [15]. Recent work by Trial Forge comparing the outcome 44 trial teams chose as their primary outcome with what patients and healthcare professionals ranked as the most important outcome found that they agreed just 30% of the time [16]. That many trial teams struggled to tell us how long it took them to collect their data strongly suggests that resource planning and budgeting does not explicitly account for data collection workload.…”

“…As mentioned previously, the ORINOCO project had three phases, and the third phase of work (not reported here) explores the process of who contributes to these discussions and how decisions about outcome selection and collection are made. There is a clear disconnect between outcome selectors (including patient and public contributors) and outcome collectors, which, at least in part, fuels the disconnect between what trialists want and what patients and health professionals want [16].…”

Patient and Public Involvement (PPI) in outcome selection in breast cancer and nephrology trials

“…We found that according to the patients and healthcare professionals, teams conducting breast cancer and nephrology trials got their choice of primary outcome wrong (72% of the time) more often than they got it right (28% of the time) [1]. A Patient and Public Involvement (PPI) representative, co-author of this letter (LL), asked (on Twitter) whether PPI contributors had been involved in the design of the original trials and by extension the outcome selection.…”

“…In a recently published study, we investigated how important patients (n = 30) and healthcare professionals (n = 12), with experience in the clinical areas of breast cancer and nephrology, consider the outcomes (particularly the primary outcome) measured in a random selection of 20 breast cancer and 24 nephrology published randomised controlled trials [1]. Primary outcomes are deemed the most important outcomes by trial investigators, and the primary outcome is used to determine the sample size for the trial [2] as well as being the main way to judge whether the intervention is effective or not.…”

“…Primary outcomes are deemed the most important outcomes by trial investigators, and the primary outcome is used to determine the sample size for the trial [2] as well as being the main way to judge whether the intervention is effective or not. Secondary outcomes aid with decision-making by providing additional information but are considered less important [1].…”