Framing the conversation: use of PRECIS-2 ratings to advance understanding of pragmatic trial design domains

Abstract

BackgroundThere continues to be debate about what constitutes a pragmatic trial and how it is distinguished from more traditional explanatory trials. The NIH Pragmatic Trials Collaborative Project, which includes five trials and a coordinating unit, has adopted the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) instrument. The purpose of the study was to collect PRECIS-2 ratings at two points in time to assess whether the tool was sensitive to change in trial design, and to explore with investigators the rationale for rating shifts.MethodsA mixed-methods design included sequential collection and analysis of quantitative data (PRECIS-2 ratings) and qualitative data. Ratings were collected at two annual, in-person project meetings, and subsequent interviews conducted with investigators were recorded, transcribed, and coded using NVivo 11 Pro for Windows. Rating shifts were coded as either (1) actual change (reflects a change in procedure or protocol), (2) primarily a rating shift reflecting rater variability, or (3) themes that reflect important concepts about the tool and/or pragmatic trial design.ResultsBased on PRECIS-2 ratings, each trial was highly pragmatic at the planning phase and remained so 1 year later in the early phases of trial implementation. Over half of the 45 paired ratings for the nine PRECIS-2 domains indicated a rating change from Time 1 to Time 2 (N = 24, 53%). Of the 24 rating changes, only three represented a true change in the design of the trial. Analysis of rationales for rating shifts identified critical themes associated with the tool or pragmatic trial design more generally. Each trial contributed one or more relevant comments, with Eligibility, Flexibility of Adherence, and Follow-up each accounting for more than one.ConclusionsPRECIS-2 has proved useful for “framing the conversation” about trial design among members of the Pragmatic Trials Collaborative Project. Our findings suggest that design elements assessed by the PRECIS-2 tool may represent mostly stable decisions. Overall, there has been a positive response to using PRECIS-2 to guide conversations around trial design, and the project’s focus on the use of the tool by this group of early adopters has provided valuable feedback to inform future trainings on the tool.

The application of PRECIS-2 ratings in randomized controlled trials of Chinese herbal medicine

“…Johnson KE [ 15 ] pointed out that evaluators struggled to use the PRECIS system for analysis, as large differences existed in inter-evaluator reliability. Furthermore, El DR et al [ 16 ] indicated that the clinical expertise of the investigator also affected scoring in each domain of PRECIS-2.…”

“…We propose several suggestions: 1) More research is urgently needed to establish the easiest and most useful tool to facilitate the applicability of results in clinical practice, distinguish between effectiveness and efficacy results and assist researchers in preparing and planning clinical trials; [ 16 ] 2) Researchers should pay attention to PRECIS-2 before they design RCTs and promote self-review during their implementation. 3) Due to the large number of Chinese researchers, the PRECIS-2 guidelines should be translated into Chinese; Related introductory articles should also be published in Chinese to promote a wider range of applications for PRECIS-2; 4) Journals all over the world that publish clinical trials should require authors to include a quantitative score related to the effectiveness or efficacy of their combined research articles; [ 16 – 18 ] 5) Several issues specific to CHM should be clarified in the new version of PRECIS. For flexibility (delivery), how do we define “a highly specified, protocol driven intervention” and “permitted co-interventions” in CHM, as doctors of Chinese medicine add or subtract herbs based on syndrome differentiation at different times?…”

Online randomized controlled experiments at scale: lessons and extensions to medicine

“…We have also learned that our postdischarge telephone calls have made no difference in rates of readmission or patient-experience ratings, that our appointment-reminder letters were completely ineffective, and that our community health worker program was inadvertently targeting patients who were unlikely to benefit" [20]. The most desirable features of A/B experiments are their large-scale and low cost, which are commensurate with the tradition of large simple trials [42] and the emerging interest in pragmatic trials [43,44]. Lower costs would allow to test more and other interventions and provide better evidence on thus far understudied healthcare questions [13,16].…”

Evaluation of the effects of pycnogenol (French maritime pine bark extract) supplementation on inflammatory biomarkers and nutritional and clinical status in traumatic brain injury patients in an intensive care unit: A randomized clinical trial protocol

“…A pragmatic [34], parallel-group, double-blind, randomized controlled trial (Table 3) will be conducted. We will enroll 60 patients who are admitted to the ICU at a university hospital in Tehran, Iran.…”