Development and psychometric evaluation of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool: a study protocol

Abstract

IntroductionThe need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research.Methods and analysisThis study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published inImplementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field.Ethics and disseminationThis study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King’s College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).

The use of implementation science theories, models, and frameworks in implementation research for medicinal products: A scoping review

“…The ImpRes specifies 10 domains or criteria to consider (although not all of the domains may be equally relevant for all IS studies): (1) implementation research characteristics, (2) implementation TMFs, (3) determinants of implementation, (4) implementation strategies, (5) service and patient outcomes, (6) implementation outcomes, (7) economic evaluation, (8) stakeholders' involvement and engagement, (9) patient and public involvement and engagement and (10) unintended consequences (Table 1). Although a comprehensive tool to appraise the conceptual and methodological quality of implementation research [Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool] is under development, no quality-appraisal tool for implementation studies currently exists [39]. In light of this gap, we opted to adapt the ImpRes for this purpose.…”

“…Although a comprehensive tool to appraise the conceptual and methodological quality of implementation research [Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool] is under development, no quality-appraisal tool for implementation studies currently exists [ 39 ]. In light of this gap, we opted to adapt the ImpRes for this purpose.…”

When a pandemic and epidemic collide: Lessons learned about how system barriers can interrupt implementation of addiction research

“…In the clinical case presented here, the urgent conditions created by the COVID-19 pandemic pre-empted changes out of necessity, and the old system was neither deconstructed nor prepared appropriately before a new approach was applied. The field of implementation science has developed critical concepts and methodological guidance that aims to support high-quality implementation research ( Proctor et al, 2011 ; Sweetnam et al, 2022 ). In retrospect, the study and its corresponding adaptations would have benefited from the use of different implementation science methods, strategies, and expertise.…”

The use of implementation science theories, models, and frameworks in implementation research for medicinal products: a scoping review

“…Research Project Appraisal Criteria [ImpResPAC] tool) is under development, no quality-appraisal tool for implementation studies currently exists [39]. In light of this gap, we opted to adapt the ImpRes for this purpose.…”