Six practical recommendations for improved implementation outcomes reporting

Abstract

Background Implementation outcomes research spans an exciting mix of fields, disciplines, and geographical space. Although the number of studies that cite the 2011 taxonomy has expanded considerably, the problem of harmony in describing outcomes persists. This paper revisits that problem by focusing on the clarity of reporting outcomes in studies that examine them. Published recommendations for improved reporting and specification have proven to be an important step in enhancing the rigor of implementation research. We articulate reporting problems in the current implementation outcomes literature and describe six practical recommendations that address them. Recommendations Our first recommendation is to clearly state each implementation outcome and provide a definition that the study will consistently use. This includes providing an explanation if using the taxonomy in a new way or merging terms. Our second recommendation is to specify how each implementation outcome will be analyzed relative to other constructs. Our third recommendation is to specify “the thing” that each implementation outcome will be measured in relation to. This is especially important if you are concurrently studying interventions and strategies, or if you are studying interventions and strategies that have multiple components. Our fourth recommendation is to report who will provide data and the level at which data will be collected for each implementation outcome, and to report what kind of data will be collected and used to assess each implementation outcome. Our fifth recommendation is to state the number of time points and frequency at which each outcome will be measured. Our sixth recommendation is to state the unit of observation and the level of analysis for each implementation outcome. Conclusion This paper advances implementation outcomes research in two ways. First, we illustrate elements of the 2011 research agenda with concrete examples drawn from a wide swath of current literature. Second, we provide six pragmatic recommendations for improved reporting. These recommendations are accompanied by an audit worksheet and a list of exemplar articles that researchers can use when designing, conducting, and assessing implementation outcomes studies.

The updated Consolidated Framework for Implementation Research based on user feedback

“…Innovation domain Innovation: The “thing” being implemented [ 20 ], e.g., a new clinical treatment, educational program, or city service Project Innovation: [Document the innovation being implemented, e.g., innovation type, innovation core vs. adaptable components, using a published reporting guideline [ 21 – 24 ]. Distinguish the innovation (the “thing” that continues when implementation is complete) [ 20 , 25 ] from the implementation process and strategies used to implement the innovation [ 26 , 27 ] (activities that end after implementation is complete) [ 28 ].] Construct name Construct definition The degree to which: A.…”

“…Capability The individual(s) has interpersonal competence, knowledge, and skills to fulfill Role C. Opportunity The individual(s) has availability, scope, and power to fulfill Role D. Motivation The individual(s) is committed to fulfilling Role V. Implementation Process domain Implementation Process: The activities and strategies used to implement the innovation Project Implementation Process: [Document the implementation process framework [ 8 ] and/or activities and strategies [ 26 , 27 ] being used to implement the innovation. Distinguish the implementation process used to implement the innovation (activities that end after implementation is complete) from the innovation (the “thing” that continues when implementation is complete) [ 20 , 25 , 28 ]. Construct name Construct definition: The degree to which individuals: A. Teaming Join together, intentionally coordinating and collaborating on interdependent tasks, to implement the innovation B. Assessing Needs Collect information about priorities, preferences, and needs of people Use this construct to capture themes related to Assessing Needs that are not included in the subconstructs below 1.…”

“…Project Implementation Process: [Document the implementation process framework [ 8 ] and/or activities and strategies [ 26 , 27 ] being used to implement the innovation. Distinguish the implementation process used to implement the innovation (activities that end after implementation is complete) from the innovation (the “thing” that continues when implementation is complete) [ 20 , 25 , 28 ].…”

“…The literature has recognized that the lack of a clean boundary between the innovation and implementation strategies is a contributor to implementation complexity [ 22 ]; however, distinguishing between the innovation and implementation strategy is necessary for accurate attribution to implementation outcomes [ 28 ] and to identify appropriate areas for future intervention. As a result, the updated CFIR guides users to define the innovation (aka “the thing” [ 20 , 25 ] being implemented), including the boundary between the innovation and implementation strategies. We encourage use of a reporting guideline to define the innovation (see Table 3 ).…”

The Updated Consolidated Framework for Implementation Research: CFIR 2.0

“…The literature has recognized that the lack of a clean boundary between the innovation and implementation strategies is a contributor to implementation complexity (33); differentiating the two helps ensure accurate attribution of constructs in each domain to implementation outcomes. As a result, the CFIR 2.0 guides users to de ne the innovation (aka "the thing" (34,35) being implemented), including the boundary between the innovation and implementation strategies. We encourage use of a reporting framework to document the innovation, such as the Workgroup for Intervention Development and Evaluation Research (WIDER) (36), PICOT/PICOTS (33,37), or the Simpli ed Framework for Interventions (AIMD) (38).…”