How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research

Abstract

BackgroundThe need for high-quality evidence that is applicable in real-world, routine settings continues to increase. Pragmatic trials are designed to evaluate the effectiveness of interventions in real-world settings, whereas explanatory trials aim to test whether an intervention works under optimal situations. There is a continuum between explanatory and pragmatic trials. Most trials have aspects of both, making it challenging to label and categorize a trial and to evaluate its potential for translation into practice.MethodsWe summarize our experience applying the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) combined with external validity items based on the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to three studies to provide a more robust and comprehensive assessment of trial characteristics related to translation of research. We summarize lessons learned using domains from the combined frameworks for use in study planning, evaluating specific studies, and reviewing the literature and make recommendations for future use.ResultsA variety of coders can be trained to use the PRECIS and RE-AIM domains. These domains can also be used for diverse purposes, content areas, and study types, but are not without challenges. Both PRECIS and RE-AIM domains required modification in two of the three studies to evaluate and rate domains specific to study type. Lessons learned involved: dedicating enough time for training activities related to the domains; use of reviewers with a range of familiarity with specific study protocols; how to best adapt ratings that reflect complex study designs; and differences of opinion regarding the value of creating a composite score for these criteria.ConclusionsCombining both frameworks can specifically help identify where and how a study is and is not pragmatic. Using both PRECIS and RE-AIM allows for standard reporting of key study characteristics related to pragmatism and translation. Such measures should be used more consistently to help plan more pragmatic studies, evaluate progress, increase transparency of reporting, and integrate literature to facilitate translation of research into practice and policy.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-014-0096-x) contains supplementary material, which is available to authorized users.

Implementation of an evidence-based intervention to promote colorectal cancer screening in community organizations: a cluster randomized trial

“…During the trial, some CHAs made modifications to the study protocol, such as referring participants to the free clinic even if they had health insurance and conducting educational sessions with individuals rather than with small groups. We did not interfere with these modifications and captured them in our assessments, as recommended for implementation research [30,31].…”

Evaluating the design and reporting of pragmatic trials in osteoarthritis research

“…Determinations were made for each criterion using a simple binary system to indicate whether the study met pragmatic criteria (yes = 1) or not (no = 0), where a maximum score of 11 could be assigned per study (Supplementary Table 2). After being trained to code [15], two independent raters (KL and KW) evaluated each study. Interrater agreement of coding for a random sample of studies (N=30) was determined to be 78%.…”

Implementation of collaborative goal setting for diabetes in community primary care

“…See Fig. 1 for the overview of the study using the expanded CONSORT model linking the RE-AIM framework, data sources, and measures used [21,29]. The University of Iowa Institutional Review Board approved all aspects of the study.…”

“…The reason for this longitudinal sample was to understand the effect of the intervention under real-world conditions, so no attempt at randomization was made [29]. Rather, recruiting a sample of patients and following them during the implementation period was meant to provide additional estimates of reach, and access to a comparison group exposed to similar clinical conditions (i.e., practice context and medical management), but not necessarily to the intervention over the 9-month study period.…”