The Elusive Search for Success: Defining and Measuring Implementation Outcomes in a Real-World Hospital Trial

Abstract

Objective and Study Setting: Research efforts to identify factors that influence successful implementation are growing. This paper describes methods of defining and measuring outcomes of implementation success, using a cluster randomized controlled trial with 12 cancer services in Australia comparing the effectiveness of implementation strategies to support adherence to the Australian Clinical Pathway for the Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients (ADAPT CP).Study Design and Methods: Using the StaRI guidelines, a process evaluation was planned to explore participant experience of the ADAPT CP, resources and implementation strategies according to the Implementation Outcomes Framework. This study focused on identifying measurable outcome criteria, prior to data collection for the trial, which is currently in progress.Principal Findings: We translated each implementation outcome into clearly defined and measurable criteria, noting whether each addressed the ADAPT CP, resources or implementation strategies, or a combination of the three. A consensus process defined measures for the primary outcome (adherence) and secondary (implementation) outcomes; this process included literature review, discussion and clear measurement parameters. Based on our experience, we present an approach that could be used as a guide for other researchers and clinicians seeking to define success in their work.Conclusions: Defining and operationalizing success in real-world implementation yields a range of methodological challenges and complexities that may be overcome by iterative review and engagement with end users. A clear understanding of how outcomes are defined and measured, based on a strong theoretical framework, is crucial to meaningful measurement and outcomes. The conceptual approach described in this article could be generalized for use in other studies.Trial Registration: The ADAPT Program to support the management of anxiety and depression in adult cancer patients: a cluster randomized trial to evaluate different implementation strategies of the Clinical Pathway for Screening, Assessment and Management of Anxiety and Depression in Adult Cancer Patients was prospectively registered with the Australian New Zealand Clinical Trials Registry Registration Number: ACTRN12617000411347.

Commentary on ‘Exclusion rates in randomized trials of treatments for physical conditions: a systematic review’

“…There is significant opportunity to improve the reporting of efficacy trials to consider future interventions and outcomes about effectiveness and potential for implementation in real-world settings. Our example of operationalising implementation outcomes used clearly defined criteria to enable measurement and systematic application across one aspect of cancer care [7]. The findings of He et al's study supports calls for greater use of pragmatic trial designs [8,9].…”

Exercise interventions for people diagnosed with cancer: a systematic review of implementation outcomes

“…This review identifies that exercise interventions are being implemented for people diagnosed with cancer using pragmatic study designs, but there is no consensus about how successful implementation should be defined, measured, and reported. Measuring implementation outcomes, using an established framework, can generate new knowledge in this area by conceptualising and defining what constitutes success [ 33 ]. To the best of our knowledge, this is the first systematic review that has explored implementation outcomes in exercise and cancer using the Implementation Outcomes Framework [ 30 ].…”

“…Evaluating the outcomes of implementation efforts can also reduce the risk of incorrectly concluding that an intervention is ineffective, when in fact, poor implementation may be the most significant contributor to failure [ 30 , 31 ]. Implementation science frameworks that evaluate implementation outcomes may therefore be useful to determine whether failure is due to the intervention or the implementation process [ 32 , 33 ]. Proctor and colleagues [ 30 ] define eight implementation outcomes for this purpose: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration and sustainability.…”

<p>Manifestation of Borderline Personality Symptomatology in Chronic Pain Patients Under Stress: An Understated and Exacerbated Consequence of the COVID-19 Crisis</p>

“…We conclude by recommending that at this particularly stressful time, the key to solving the BPD and chronic pain dilemma is more routine screening for personality disorders among all patients receiving pain management services. With progressive frequency, physicians who treat patients with pain are administering screening measures for depression and anxiety, with recent recommendations made for such screening among patients in chronic pain management involving opioid therapy, 75 prior to spine surgery, 76 in cancer care, 77 and in primary care practice, generally. [78][79][80] Further, some suggest that more frequent urine drug testing (UDT) is actually an answer to the opioid misuse problem in the pain management setting, 81 with recommendations that all patients receiving chronic opioid therapy undergo monthly UDTnot to exclude patients from treatmentbut rather to determine which patients require a higher degree of monitoring in order to optimize treatment safety and efficacy.…”