Evaluating impact of clinical guidelines using a realist evaluation framework

Abstract

Implementing a realist evaluation framework to generate robust findings in this context has required innovation in the evaluation design and adaptation by researchers. This article captures the RPHCM team's experience in designing this evaluation.

Use of clinical guidelines in remote Australia: A realist evaluation

“…The evaluation framework allows for this theory to be tested and then confirmed or revised. Details of the realist evaluation protocol have been previously published in this journal . The published article outlines the process involved in implementing this evaluation.…”

“…The guidance for formulating preliminary realist programme theory is to review previous research findings, literature, and converse with stakeholders. The authors on the basis of this recommendation used previous evaluation findings and feedback from RPHCM users and other stakeholders to construct the preliminary programme theory . This theory needed to operate in the context of the challenges of remote health services such as isolation and workforce turnover.…”

“…This theory needed to operate in the context of the challenges of remote health services such as isolation and workforce turnover. The preliminary theory was as follows:

Remote health practitioners have adopted standardised, evidence‐based practice because of the use of Remote Primary Health Care Manuals (RPHCM). The mechanisms that lead to the use of the manuals include fear of consequences of not using RPHCM, confidence in using and understanding RPHCM content, reliance on manual content to guide their practice, the perception of credibility, and applicability of RPHCM content to the context.

Some remote health practitioners regularly use the RPHCM in their daily practice because there is a lack of alternative resources for RPHCM content.

Some remote health practitioners are inclined to use the RPHCM less regularly because of a perception that the content is less suited to their needs and daily practice.

…”

“…The programme theory, which had been outlined as a flow diagram in the publication about the protocol, is reproduced here (see Figure ).…”

“…The evaluators then proceeded to test the programme theory. As this is a theory‐based evaluation assessing a complex scenario, a variety of methods were used to collect data: face‐to‐face interviews, telephone interviews, online survey, and clinical audits . This approach aligns with realist evaluation, which favours a combination of methods to test the programme theory.…”