Comparing, Contrasting, and Integrating Dissemination and Implementation Outcomes Included in the RE-AIM and Implementation Outcomes Frameworks

Abstract

As the field of dissemination and implementation science matures, there are a myriad of outcomes, identified in numerous frameworks, that can be considered across individual, organizational, and population levels. This can lead to difficulty in summarizing literature, comparing across studies, and advancing translational science. This manuscript sought to (1) compare, contrast, and integrate the outcomes included in the RE-AIM and Implementation Outcomes Frameworks (IOF) and (2) expand RE-AIM indicators to include relevant IOF dissemination and implementation outcomes. Cross tabular comparisons were made between the constitutive definitions of each construct, across frameworks, to reconcile apparent discrepancies between approaches and to distinguish between implementation outcomes and implementation antecedents. A great deal of consistency was identified across approaches, including adoption (the intention, initial decision, or action to employ an evidence-based intervention), fidelity/implementation (the degree to which an intervention was delivered as intended), organizational maintenance/sustainability (extent to which a newly implemented treatment is maintained or institutionalized), and cost. The IOF construct of penetration was defined as a higher-order construct that may encompass the reach, adoption, and organizational maintenance outcomes within RE-AIM. Within the IOF approach acceptability, appropriateness, and feasibility did not match constitutive definitions of dissemination or implementation but rather reflected theoretical antecedents of implementation outcomes. Integration of the IOF approach across RE-AIM indicators was successfully achieved by expanding the operational definitions of RE-AIM to include antecedents to reach, adoption, implementation, and organizational maintenance. Additional combined metrics were also introduced including penetration, individual level utility, service provider utility, organizational utility, and systemic utility. The expanded RE-AIM indicators move beyond the current approaches described within both the RE-AIM framework and IOF and provides additional planning and evaluation targets that can contribute to the scientific field and increase the translation of evidence into practice.

Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum

“…These measures can be used to predict any anticipated or actual implementation outcome; for example, Weiner et al developed measures for acceptability, appropriateness, and feasibility of an innovation and highlighted their role as potential predictors of adoption or implementation [ 27 ]. Thus, like Reilly et al, we classify these measures as “Antecedent Assessments” [ 28 ]. Additionally, the CFIR lists implementation climate and implementation readiness as higher-order constructs within the framework—each comprising multiple determinants.…”

“… a Maintenance (setting-level): The extent to which “a program or policy becomes institutionalized or part of the routine organizational practices and policies. Includes proportion and representativeness of settings that continue the intervention and reasons for maintenance, discontinuance or adaptation.” c Sustainability: “The extent to which a newly implemented treatment is maintained or institutionalized within a service setting’s ongoing, stable operations.” Actual Implementation Outcomes: Sustainment Setting Impact (setting-level): Adoption X Implementation [ 28 , 32 ] N/A: Not explicitly included N/A: Not explicitly included Reach (recipient-level): “ The absolute number, proportion, and representativeness of individuals who are willing to participate in a given initiative, intervention, or program.” b Penetration (recipient-level): “The number of eligible persons who use a service, divided by the total number of persons eligible for the services.” Innovation Outcomes: Innovation Impact on Recipients, Deliverers, and Key Decision-Makers a Implementation (recipient-level) “Clients’ use of the intervention and implementation strategies.” Effectiveness: “The impact of an intervention on important outcomes, including potential negative effects, quality of life, and economic outcomes.” e Client Outcomes: “Satisfaction, Function, and Symptomology.” a Maintenance (recipient-level): The extent to which “behavior is sustained 6 months or more after treatment or intervention.” d Recipient Impact (recipient-level): Reach X Effectiveness [ 28 , 32 ] N/A: Not explicitly included N/A: Not explicitly included e Service Outcomes: “The extent to which services are safe, effective, patient-centered, timely, efficient, and equitable.” a Implementation and Maintenance: The RE-AIM framework includes definitions for Implementation and Maintenance at (1) the setting-level, which map to our Implementation Outcomes and at (2) the innovation recipient-level, which map to our Innovation Outcomes b Penetration: The IOF provides a definition for Penetration at (1) the deliverer-level, which maps to our Implementation Outcomes and at (2) the recipient-level, which maps to our Innovation Outcomes c Sustainability: Though the IOF uses the word Sustainability, the definition of this outcome maps to Sustainment in the CFIR Outcomes Addendum d The CFIR Outcomes Addendum conceptualizes Recipient Impact for all constituents, which can be measured via Reach × Effectiveness [ 32 ] and Reach × Maintenance together [ ...…”

Dissemination and Implementation Science Approaches for Occupational Safety and Health Research: Implications for Advancing Total Worker Health

“…As with any developing field, there are minor differences in terminology and a relative emphasis on different theories, models and frameworks. However, within D&I, there is general agreement on key principles, factors, and methods (see Table 1; also [66]).…”

“…While there have been a large number of D&I TMFs proposed, leading some scholars to perceive the field as a "Tower of Babel" [65], there are more commonalities than differences across TMFs on factors known to affect the adoption, implementation, and sustainment of evidence-based interventions [66]. Key constructs commonly considered include multilevel contexts (i.e., inner and outer contexts with multiple layers within each, as described above) [8,9,56,61], characteristics of recipients at multiple levels [9,55], intervention and implementation strategy characteristics, and considerations for the various stages or phases of D&I (e.g., exploration, preparation, reach, adoption, implementation, sustainment, and important outcomes) [8,9,56,61].…”

Implementation Strategies Preferred by Primary Care Clinicians to Facilitate Cancer Prevention and Control Activities

“…This research is an important early step in identifying implementation strategies likely to be most useful for delivering evidence-based CPC services in primary care. Other steps could include 1) evaluating if actual 'observed feasibility' (implementation consistency) and impact (effectiveness) are similar to these clinician ratings of projected impact and feasibility (32,33) and 2) investigating additional ERIC strategies (12,22) or those created by practice teams. Implementing and studying CPC activities in primary care is a challenging undertaking, including complexities such as that context, personnel, competing demands, and implementation strategies often change over time, (34) and strategies are often used in combinations or strategy bundles.…”