Using Logic Models and the Action Model/Change Model Schema in Planning the Learning Community Program: A Comparative Case Study

Abstract

Recent interest has been noted in the evaluation community in expanding the focus from program implementation and outcomes to program design and planning. One important step for moving in this direction is to examine existing evaluation models and to assess their relative strengths and weaknesses for planning purposes. This article presents a comparative case study of applying logic models and the action model/change model schema for planning the Learning Community Program in Taiwan. Lessons learned from these applications indicate that logic models are relatively easy to learn and effective for identifying major program components and indicators, but not sufficient for articulating the theoretical significance of the program. On the other hand, the action model/change model schema requires more time to learn and practise, but it has relative advantages for providing theoretical insights into contextual factors and causal mechanisms of the program, unlike logic models. This comparison can serve as a guide for evaluation practitioners when selecting evaluation tools to apply in planning and/or evaluating their programs. Résumé : De plus en plus les évaluateurs semblent orienter leur intérêt vers la conception et la planification de programme plutôt que sur l'implantation et les effets. Une étape importante de ce virage est d' examiner les modèles d' évaluation existants et d' évaluer leurs points forts et leurs points faibles pour l' exercice de planification. Cet article présente une étude de cas comparative de l'utilisation des modèles logiques et des modèle d' action/de changement aux fins de la planification d'un programme d' apprentissage communautaire à Taiwan. Nos résultats indiquent que les modèles© 2018 Canadian Journal of Program Evaluation / La Revue canadienne d'évaluation de programme 33.1 (Spring / printemps), 49-68 figure 1. Action Model/Change Model Schema Using Logic Models and the Action Model/Change Model Schema 57 CJPE 33.1, 49-68 variety of items. This is especially true for the activity component. It seems to me anything we do can be put in this component. In this case, the activity component could become a garbage can. I think the schema overcomes this problem by differentiating components between the action model and change model.

Participatory action research to pilot a model of mental health service user involvement in an Ethiopian rural primary healthcare setting: study protocol

“…To the best of our knowledge, no one has yet offered a description of how to combine ToC and PAR to apply service-user involvement in mental health systems. We attempted to address this gap by taking cues from the evidence base on combining ToC and PAR from other disciplines (e.g., agriculture [100,108], education [102], development studies [109], programme evaluation [110,111] and implementation sciences [112].…”

“…Both anticipate a range of positive changes/outcomes [96,100] including: individual level outcomes(e.g., advancing participants' personal and collective sense of agency, social networks) and community level outcomes(awareness raising, stigma reduction, and strengthening community capacity, collaboration) [54,56,113]. ToC provides a strong heuristic device for deeper understanding of the implementation context [100,112,114], guides the direction of change and how to achieve the intended transformation, defines collaborative outcomes and surfaces the various layers of interventions with underlying assumptions, and ensures that different perspectives of participants are reflected in the design [111,115,116]. However, that alone may not be sufficient to support the actions required to achieve implementation [102,107,108].…”

Does Your Implementation Fit Your Theory of Change?

“…Such elements and assumptions are rarely included in theory of change depictions and logic models, yet experienced educational evaluators (and empirical evidence) suggest that delivery and design components can make a huge difference to the success of educational investments. The quality of teaching is particularly important; see, for example, Chetty, Friedman, and Rockoff (2014), whose statistical study sug gests that teacher quality is associated with huge differences in student outcomes.…”

“…As an extension of Chen's comments, see also Sager and Andereggen (2012), Mayne and Stern (2013), Montague and Porteous (2013), Renger, Bartel, and Foltysova (2013), and the six distinguishing elements of an action model (Chen et al, 2018). Indeed, Chen's program-theory models typically show reasonably elaborate action theories.…”

“…According to Rogers (2014), a "theory of change" es sentially explains how activities are understood to produce a series of results that contribute to intended impacts. Chen, Pan, Morosanu, and Turner (2018) have recently gone on to distinguish the theory of change and the action theory quite definitively. They note that the change model describes the causal process gener ated by the program and distinguish the action model from the change model as a "systematic plan for arranging staff, resources, settings and support organizations, to reach a target group and deliver intervention services" (p. 54).…”