Does Your Implementation Fit Your Theory of Change?

Montague, Steve

doi:10.3138/cjpe.53008

Cited by 7 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hence, the ToC development process by itself can be capacity building [ 66 ], helping to foster learning and reflection [ 67 ], and can facilitate contextualised solutions through a sense of stakeholder ownership of the programme [ 68 – 70 ]. Particularly when ToC employs the techniques of participatory action research, it can serve as a catalyst for learning and promote informed action by supporting stakeholders to achieve sustained positive changes in the local context [ 35 , 40 , 71 ].…”

Section: Discussionmentioning

confidence: 99%

Developing a Theory of Change model of service user and caregiver involvement in mental health system strengthening in primary health care in rural Ethiopia

Abayneh

Lempp

Alem

et al. 2020

Int J Ment Health Syst

View full text Add to dashboard Cite

Background The involvement of service users and caregivers is recommended as a strategy to strengthen health systems and scale up quality mental healthcare equitably, particularly in low-and-middle-income countries. Service user and caregiver involvement is complex, and its meaningful implementation seems to be a worldwide challenge. Theory of Change (ToC) has been recommended to guide the development, implementation and evaluation of such complex interventions. This paper aims to describe a ToC model for service user and caregiver involvement in a primary mental health care in rural Ethiopia. Methods The ToC was developed in two workshops conducted in (i) Addis Ababa with purposively selected psychiatrists (n = 4) and multidisciplinary researchers (n = 3), and (ii) a rural district in south-central Ethiopia (Sodo), with community stakeholders (n = 24). Information from the workshops (provisional ToC maps, minutes, audio recordings), and inputs from a previous qualitative study were triangulated to develop the detailed ToC map. This ToC map was further refined with written feedback and further consultative meetings with the research team (n = 6) and community stakeholders (n = 35). Results The experiential knowledge and professional expertise of ToC participants combined to produce a ToC map that incorporated key components (community, health organisation, service user and caregiver), necessary interventions, preconditions, assumptions and indicators towards the long-term outcomes. The participatory nature of ToC by itself raised awareness of the possibilities for servicer user and caregiver involvement, promoted co-working and stimulated immediate commitments to mobilise support for a grass roots service user organization. Conclusions The ToC workshops provided an opportunity to co-produce a ToC for service user and caregiver involvement in mental health system strengthening linked to the planned model for scale-up of mental health care in Ethiopia. The next steps will be to pilot a multi-faceted intervention based on the ToC and link locally generated evidence to published evidence and theories to refine the ToC for broader transferability to other mental health settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Developing a Theory of Change model of service user and caregiver involvement in mental health system strengthening in primary health care in rural Ethiopia

Abayneh

Lempp

Alem

et al. 2020

Int J Ment Health Syst

View full text Add to dashboard Cite

show abstract

“…Second, although many programme theories articulate intended changes [98], engage with the complexity of interventions and provide a framework to guide action, monitoring and evaluation [99], there is little empirical evidence of how theories can be applied in practice [100][101][102]. Hence, there have been calls for PAR in theory-based implementation and evaluations [103][104][105]. Early integration of Theory of Change (ToC) and PAR during planning and implementation is recommended [103,106,107].…”

Section: Designmentioning

confidence: 99%

Participatory action research to pilot a model of mental health service user involvement in an Ethiopian rural primary healthcare setting: study protocol

2020

View full text Add to dashboard Cite

Background: Involvement of service-users at all levels of the mental health system is a policy imperative in many countries internationally. However, putting policy into practice seems complex; little is known about how best to involve service users and efforts are often criticized for being tokenistic. In low-and-middle income countries, less attention has been given to the roles of service users within mental health systems. The proposed study is part of a larger project intended to develop service-user involvement in mental health system strengthening in Ethiopia. A Theory of Change (ToC) model has already been developed through a participatory approach. This study protocol aims to describe the theoretical background and methods to pilot this model using participatory action research (PAR) and explore participants' experience of involvement. Methods: The proposed study will apply a PAR approach situated in critical social theory and conduct a phenomenological case study to find out participants' experience of involvement. This will be conducted in three stages. The focus of Stage 1 will be to(i) establish a Research Advisory Group (RAG), and Research Participant Group (RPG) at district and primary healthcare facility levels, respectively, and (ii) identify and prioritize potential areas of concern for involvement in the domains of advocacy, service planning and development, monitoring and improving service quality. In Stage 2, we will work with the RPG to develop a plan of action for the selected area. Stage 3 will aim to assist the RPG to implement and evaluate the plan of action. Process indicators and observation will be combined with in-depth interviews with participants to elicit their experiences of involvement. Thematic content analysis will be used. Discussion: The participatory approach to mental health service user involvement in health system strengthening employed by this study will support the implementation of solutions through locally relevant and contextualized actions. Findings from this study will contribute to the body of knowledge towards understanding the complexity of implementation of service user involvement and refine the ToC model for transferability to similar settings.

show abstract

“…29,37,38 In such studies, the evaluation can explore theoretically grounded questions about programme change mechanisms, pathways to impact or programme outcomes even as the evaluation scope precludes the examination of a programme's actual degree of implementation. 29,38 Indeed, conducting an evaluation of a programme that has not been adequately implemented has been labelled a 'Type III error.' 39,40 Notably, medical education scholars have argued recently that evaluative studies in our field need to do a better job of examining programme features and studying 'actual' implementation in context in order to generate meaningful knowledge about programme impact.…”

Section: E Valuati On For Cultivating Knowledg E and Under S Tandinmentioning

confidence: 99%

“…The risk here is in attending to novelty at the expense of performing the work required to verify more routine aspects of programme implementation. For example, evaluation scholars have raised concerns about researchers executing evaluation studies about programme theories without addressing the ‘primary question’ about the actual existence of the programme being evaluated 29,37,38 . In such studies, the evaluation can explore theoretically grounded questions about programme change mechanisms, pathways to impact or programme outcomes even as the evaluation scope precludes the examination of a programme’s actual degree of implementation 29,38 .…”

Section: Evaluation For Cultivating Knowledge and Understanding: Concmentioning

confidence: 99%

“…For example, evaluation scholars have raised concerns about researchers executing evaluation studies about programme theories without addressing the ‘primary question’ about the actual existence of the programme being evaluated 29,37,38 . In such studies, the evaluation can explore theoretically grounded questions about programme change mechanisms, pathways to impact or programme outcomes even as the evaluation scope precludes the examination of a programme’s actual degree of implementation 29,38 . Indeed, conducting an evaluation of a programme that has not been adequately implemented has been labelled a ‘Type III error.’ 39,40 …”

Section: Evaluation For Cultivating Knowledge and Understanding: Concmentioning

confidence: 99%

See 1 more Smart Citation

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

Onyura

2020

Medical Education

View full text Add to dashboard Cite

Context: Programme evaluation is perpetually mandated in health professions education. Correspondingly, there has been an expansion in prescriptive methodological guides about 'how' to engage in various best practices in evaluation. However, what has gained less attention is an examination of the 'value' that different stakeholders seek to gain from programme evaluation. Evaluation utilisation theory and research can help us understand the diversity in both the driving forces for and the impact of programme evaluation. Awareness of the heterogeneity of evaluation utilisation priorities has implications for evaluation practices, including both methodological choices and understanding of the impact of programme evaluation in our field. Methods: In this article, I expound on the concept of evaluation utilisation by drawing on evaluation theory and research. 'Evaluation utilisation' refers to the application of programme evaluation processes and findings to influence thinking and action. Herein, different forms of 'evaluation utilisation' (including instrumental, conceptual, process and persuasive (symbolic and legitimative) utilisation) are discussed, as well as the related concept of 'evaluation misuse'. Furthermore, how the prioritisation of different forms of 'evaluation utilisation' can influence the scope and impact of evaluation scholarship are also discussed. Conclusions: Programme evaluation is a form of inquiry that requires more than the exercise of robust methodological techniques. Rather, it necessitates attention to the, sometimes divergent, priorities of different stakeholder groups. Although there is scant research on evaluation practices in health professions education, evaluation utilisation theory can inform critical examination of evaluation practices and impact in our field. Critically, understanding this body of work can help inform those engaged in evaluation about what they are (or should be) prioritising when they conduct programme evaluation, and better align evaluation methodologies with their scholarly, curricular and administrative intentions. Implications for future research and high-quality, transparent evaluation scholarship are presented. How to cite this article: Onyura B. Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope.

show abstract

Does Your Implementation Fit Your Theory of Change?

Cited by 7 publications

References 7 publications

Developing a Theory of Change model of service user and caregiver involvement in mental health system strengthening in primary health care in rural Ethiopia

Developing a Theory of Change model of service user and caregiver involvement in mental health system strengthening in primary health care in rural Ethiopia

Participatory action research to pilot a model of mental health service user involvement in an Ethiopian rural primary healthcare setting: study protocol

Useful to whom? Evaluation utilisation theory and boundaries for programme evaluation scope

Contact Info

Product

Resources

About