Verifying participant-reported clinical outcomes: challenges and implications

Abstract

Background: Researchers often rely on trial participants to self-report clinical outcomes (for example, fractures, reoperations). Little information exists as to the 'accuracy' of participant-reported clinical outcomes, particularly in randomised controlled trials (RCTs). To help address this evidence gap, we report four case studies, nested within different RCTs where participant-reported clinical outcome data were compared with those reported by clinicians or extracted from medical notes. Methods: Four publicly-funded RCTs with different methods of verifying participant-reported outcomes were identified. In KAT, the participants were asked about hospital admissions for any reason. Where it was thought to be relevant to the trial knee, further information was sought from the lead surgeon at the admitting site to confirm whether or not the admission was relevant to the trial knee. In REFLUX, participants were asked about hospital admissions for any reason. For participants who reported a re-operation, further information was sought from the lead surgeon at the admitting site to confirm this. In RECORD, participants were asked three questions regarding broken bones. Where low-trauma fractures were reported, clinical verification was sought, initially from the research nurse at the site. In CATHETER, participants were asked about urinary tract infections (UTIs), and a prescription of antibiotics was provided for the treatment of UTIs following urethral catheterisation. The GPs of those who reported a UTI were contacted to confirm that an antibiotic prescription had been issued for the suspected UTI. Results: In KAT, 397 of 6882 (6%) participant-reported hospital admissions were confirmed as relevant to the trial knee. In REFLUX, 16 of 19 participants (84%) who appeared to have had a re-operation were confirmed as having had one. In RECORD, 473 of 781 (61%) fractures reported by participants were confirmed as being low-trauma fractures. In CATHETER, 429 of 830 participant-reported UTIs (52%) were confirmed as such by the GPs. Conclusions: We used different approaches in our verification of participant-reported outcomes in clinical trials, and we believe there is no one optimal solution. Consideration of issues such as what information is sought from participants, the phrasing of questions, whether the medical records are a true 'gold standard' and costs and benefits to the RCT may help determine the appropriate approach.

Concordance between survey and electronic health record data in the COVID-19 Citizen Science study: a retrospective cohort analysis (Preprint)

“…In parallel, implementation of best-practices to enhance validity of participant-report, including stakeholder engagement in survey design, readability assessments, and cultural and linguistic adaptation are essential to enhancing reliability of findings from participant-facing research. [26][27] Our findings are broadly consistent with those from prior studies suggesting that fitness-for-use depends on context. [28][29] We found that no single data source may be appropriate for EHRbased pragmatic research, consistent with prior work illustrating the potential biases that can arise in participant-reported data and how they vary.…”