Managing Tensions Between Evaluation and Research

Rey, Lynda; Tremblay, Marie‐Claude; Brousselle, Astrid

doi:10.1177/1098214013503698

Cited by 32 publications

(11 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Our prior experience in action research may have helped to prevent potential role tension. Rey et al [5] liken a DE approach to conducting action research, explaining that DE evaluators engage in experiential learning cycles to both produce knowledge and facilitate change. In spanning the boundary between researchers and stakeholders, our evaluator helped to achieve the project’s knowledge co-production aim.…”

Section: Discussionmentioning

confidence: 99%

“…Developmental evaluation (DE) is a growing area of evaluation practice, developed to accommodate emergent programmes and projects. DE is used to inform adaptive development of change initiatives in complex environments [1–3]; however, there is limited literature describing its use in Australian Aboriginal and Torres Strait Islander (hereafter respectfully referred to as Indigenous) health programmes [4] or in knowledge translation research [5, 6]. This article is based on our experience of using DE to support the implementation of a theory-informed process defined as ‘interactive dissemination’.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Laycock

Bailie

Matthews

et al. 2019

Health Res Policy Sys

View full text Add to dashboard Cite

Background Developmental evaluation is a growing area of evaluation practice, advocated for informing the adaptive development of change initiatives in complex social environments. The utilisation focus, complexity perspective and systems thinking of developmental evaluation suggest suitability for evaluating knowledge translation initiatives in primary healthcare. However, there are few examples in the literature to guide its use in these contexts and in Indigenous settings. In this paper, we reflect on our experience of using developmental evaluation to implement a large-scale knowledge translation research project in Australian Aboriginal and Torres Strait Islander primary healthcare. Drawing on principles of knowledge translation and key features of developmental evaluation, we debate the key benefits and challenges of applying this approach to engage diverse stakeholders in using aggregated quality improvement data to identify and address persistent gaps in care delivery. Discussion The developmental evaluation enabled the team to respond to stakeholder feedback and apply learning in real-time to successfully refine theory-informed research and engagement processes, tailor the presentation of findings to stakeholders and context, and support the project’s dissemination and knowledge co-production aim. It thereby contributed to the production of robust, useable research findings for informing policy and system change. The use of developmental evaluation appeared to positively influence stakeholders’ use of the project reports and their responses to the findings. Challenges included managing a high volume of evaluation data and multiple evaluation purposes, balancing facilitative sense-making processes and change with task-focused project management, and lack of experience in using this evaluation approach. Use of an embedded evaluator with facilitation skills and background knowledge of the project helped to overcome these challenges, as did similarities observed between features of developmental evaluation and continuous quality improvement. Conclusion Our experience of developmental evaluation confirmed our expectations of the potential value of this approach for strengthening improvement interventions and implementation research, and particularly for adapting healthcare innovations in Indigenous settings. In our project, developmental evaluation successfully encompassed evaluation, project adaptation, capacity development and knowledge translation. Further work is warranted to apply this approach more widely to improve primary healthcare initiatives and outcomes, and to evaluate implementation research.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Using developmental evaluation to support knowledge translation: reflections from a large-scale quality improvement project in Indigenous primary healthcare

Laycock

Bailie

Matthews

et al. 2019

Health Res Policy Sys

View full text Add to dashboard Cite

show abstract

“…We will explore whether a rapid analytical procedure using analytical memos might overcome some challenges encountered by other researchers [41]. We will also discover whether guideline development teams and site implementation teams experience DE as having a positive (or negative) impact on their efforts to develop and implement deprescribing guidelines.…”

Section: Discussionmentioning

confidence: 99%

“…DE has been used to reveal program impacts [35], to facilitate social innovation through public health interventions [41], to reveal team dynamics that promote program development [42], to facilitate discussion of the impact of research [43], and to facilitate change through team dialog [44]. …”

Section: Using Developmental Evaluation To Address Challenges Relatedmentioning

confidence: 99%

See 1 more Smart Citation

Developmental evaluation as a strategy to enhance the uptake and use of deprescribing guidelines: protocol for a multiple case study

et al. 2015

View full text Add to dashboard Cite

BackgroundThe use of developmental evaluation is increasing as a method for conducting implementation research. This paper describes the use of developmental evaluation to enhance an ongoing study. The study develops and implements evidence-based clinical guidelines for deprescribing medications in primary care and long-term care settings. A unique feature of our approach is our use of a rapid analytical technique.Methods/DesignThe team will carry out two separate analytical processes: first, a rapid analytical process to provide timely feedback to the guideline development and implementation teams, followed by a meta-evaluation and second, a comprehensive qualitative analysis of data after the implementation of each guideline and a final cross-case analysis. Data will be gathered through interviews, through observational techniques leading to the creation of field notes and narrative reports, and through assembling team documents such as meeting minutes. Transcripts and documents will be anonymized and organized in NVIVO by case, by sector (primary care or long-term care), and by implementation site. A narrative case report, directed coding, and open coding steps will be followed. Clustering and theming will generate a model or action map reflecting the functioning of the participating social environments.DiscussionIn this study, we will develop three deprescribing guidelines and will implement them in six sites (three family health teams and three long-term care homes), in a sequential iterative manner encompassing 18 implementation efforts. The processes of 11 distinct teams within four conceptual categories will be examined: a guideline priority-setting group, a guideline development methods committee, 3 guideline development teams, and 6 guideline implementation teams. Our methods will reveal the processes used to develop and implement the guidelines, the role and contribution of developmental evaluation in strengthening these processes, and the experience of six sites in implementing new evidence-based clinical guidelines. This research will generate new knowledge about team processes and the uptake and use of deprescribing guidelines in family health teams and long-term care homes, with a goal of addressing polypharmacy in Canada. Clinicians and researchers creating clinical guidelines to introduce improvements into daily practice may benefit from our developmental evaluation approach.Electronic supplementary materialThe online version of this article (doi:10.1186/s13012-015-0279-0) contains supplementary material, which is available to authorized users.

show abstract