Building a framework for the evaluation of knowledge translation for the Canadian Network for Observational Drug Effect Studies

Abstract

PurposeThe Canadian Network for Observational Drug Effect Studies (CNODES), a network of pharmacoepidemiologists and other researchers from seven provincial sites, provides evidence on the benefits and risks of drugs used by Canadians. The Knowledge Translation Team, one of CNODES' four main teams, evaluates the impact of its efforts using an iterative and emergent approach. This article shares key lessons from early evaluation phases, including identifying stakeholders and their evaluation needs, choosing evaluation theories and approaches, and developing evaluation questions, designs, and methods appropriate for the CNODES context.MethodsStakeholder analysis was conducted using documentary analysis to determine key contextual factors and research evidence needs of decision maker partners and other stakeholders. Selected theories and frameworks from the evaluation and knowledge translation literature informed decisions about evaluation design and implementation. A developmental approach to evaluation was deemed appropriate due to the innovative, complex, and ever‐changing context.ResultsA theory of change, logic model, and potential evaluation questions were developed, informed by the stakeholder analysis. Early indicators of program impact (citation metrics, alternative metrics) have been documented; efforts to collect data on additional indicators are ongoing.ConclusionA flexible, iterative, and emergent evaluation approach allows the Knowledge Translation Team to apply lessons learned from completed projects to ongoing research projects, adapt its approaches based on stakeholder needs, document successes, and be accountable to funders/stakeholders. This evaluation approach may be useful for other international pharmacoepidemiology research networks planning and implementing evaluations of similarly complex, multistakeholder initiatives that are subject to constant change.

Exploring parity in female authorship of pharmacoepidemiology articles: A case study of the Canadian Network for Observational Drug Effect Studies and its citing articles

“…Generally the project lead, each site lead, and the lead analyst are authors, as specified in CNODES' guides 38 . As part of CNODES' evaluation, 39 our objective was to determine the proportion and position of female authorships in CNODES articles and to compare those to the rate in the general pharmacoepidemiology literature.…”

Data sources for drug utilization research in Latin American countries—A cross‐national study: DASDUR‐LATAM study

“…In Canada, the Canadian Network for Observational Drug Effect Studies has similar goals. The European Union has recently launched a Big Data taskforce to address this issue 22–24 …”

Five ways to get a grip on the drawbacks of logic models in program evaluation

“…For example, nested logic models incorporate multiple logic models that are linked through shared activities or outcomes and thus 'nested' into one another. 20 They compartmentalize the structural complexity of multiprogram systems: the highest level model of a nested logic model appears concise, but is constituted by multiple, aligned 'sub-logic models.' 2,20 Action models are another contemporary option that are more expansive than traditional logic models.…”

“…20 They compartmentalize the structural complexity of multiprogram systems: the highest level model of a nested logic model appears concise, but is constituted by multiple, aligned 'sub-logic models.' 2,20 Action models are another contemporary option that are more expansive than traditional logic models. In addition to articulating pathways via which a program may influence specified outcomes, action models outline the prescriptive assumptions about how those pathways may be activated.…”