A Pragmatic Evaluation of a Public Health Knowledge Broker Mentoring Education Program: A Convergent Mixed Methods Study

Abstract

Background: Public health professionals are expected to use the best available research and contextual evidence to inform decision making. The National Collaborating Centre for Methods and Tools developed, implemented, and evaluated a Knowledge Broker mentoring program aimed at facilitating organization-wide evidence-informed decision making in ten public health units in Ontario, Canada. The purpose of this study was to pragmatically assess the impact of the program. Methods: A convergent mixed methods design was used to interpret quantitative results in the context of the qualitative findings. Quantitatively, participants’ knowledge and skills for finding, interpreting, and using evidence were measured before and after program completion via multiple-choice tests. Changes in scores were assessed using paired t-tests. Qualitatively, program participants and management at enrolled public health units were interviewed to explore the effect of program participation. A secondary analysis of these interviews was conducted to determine whether organizations met their evidence use goals set at baseline, and to identify key factors related to implementation of EIDM within the organization. Results: Post-program scores for knowledge and skills for EIDM were higher compared to pre-program scores (mean difference = 14.0%, 95% CI 8.2%, 19.8%). Organizations met their goals for evidence use to varying degrees. Key themes identified that support an organizational shift to EIDM include definitive plans for participants to share knowledge during and after program completion, embedding evidence into decision making processes, and supportive leadership with organizational investment of time and resources. The location, setting or size of health units was not associated with attainment of EIDM goals; small, rural health units were not at a disadvantage compared to larger, urban health units. Conclusions: The Knowledge Broker mentoring program effectively increased participants’ knowledge and skill, allowing them to share their learning and support change at their health units. When paired with organizational supports such as supportive leadership and resource investment, this program holds promise as an innovative knowledge translation strategy for organization wide EIDM among public health organizations.

Practices Among Local Public Health Agencies to Support Evidence-Based Decision Making: A Qualitative Study

“…37,38 Interview participants strongly expressed wanting more staff members to receive the 3 1 2 -day training and additional step-bystep support. To build LHD capacity, other academic research teams have provided embedded knowledge brokers in local public health agencies 39,40 ; leadership coaching 41 ; change management training by the National Association of County and City Health Officials (available at: https://www.pathlms.com/naccho/ courses/24251) 7,42,43 ; broad leadership training 44,45 ; or support of LHD establishment of broad ongoing partnerships with nearby universities and colleges. 46,47 In-person or remote technical assistance can facilitate organizational supports for EBDM and EBDM use but is difficult to sustain without grant funding.…”