Programme evaluation training for health professionals in francophone Africa: process, competence acquisition and use

Abstract

BackgroundWhile evaluation is, in theory, a component of training programmes in health planning, training needs in this area remain significant. Improving health systems necessarily calls for having more professionals who are skilled in evaluation. Thus, the Université de Ouagadougou (Burkina Faso) and the Université de Montréal (Canada) have partnered to establish, in Burkina Faso, a master's-degree programme in population and health with a course in programme evaluation. This article describes the four-week (150-hour) course taken by two cohorts (2005–2006/2006–2007) of health professionals from 11 francophone African countries. We discuss how the course came to be, its content, its teaching processes and the master's programme results for students.MethodsThe conceptual framework was adapted from Kirkpatrick's (1996) four-level evaluation model: reaction, learning, behaviour, results. Reaction was evaluated based on a standardized questionnaire for all the master's courses and lessons. Learning and behaviour competences were assessed by means of a questionnaire (pretest/post-test, one year after) adapted from the work of Stevahn L, King JA, Ghere G, Minnema J: Establishing Essential Competencies for Program Evaluators. Am J Eval 2005, 26(1):43–59. Master's programme effects were tested by comparing the difference in mean scores between times (before, after, one year after) using pretest/post-test designs. Paired sample tests were used to compare mean scores.ResultsThe teaching is skills-based, interactive and participative. Students of the first cohort gave the evaluation course the highest score (4.4/5) for overall satisfaction among the 16 courses (3.4–4.4) in the master's programme. What they most appreciated was that the forms of evaluation were well adapted to the content and format of the learning activities. By the end of the master's programme, both cohorts of students considered that they had greatly improved their mastery of the 60 competences (p < 0.001). This level was maintained one year after completing the master's degree, except for reflective practice (p < 0.05). Those who had carried out an evaluation in the intervening 12 months reported a negative gap between their declared mastery and their actual application. However, this is only statistically significant for reflective practice (p < 0.05).ConclusionThis study shows the importance of integrating summative evaluation into the learning process. Skills-based teaching is much appreciated and well-adapted. Creating a master's programme in population and health in Africa and providing training in evaluation to high-level health professionals from many countries augurs well for scaling up the practice of evaluation in African health systems.

Family Health Strategy professionals' view on the effects of Hansen's disease training

“…Studies on the effectiveness of training conducted in other contexts show the importance of support during transfer of learning, encouraging reflection on their own performance and the exchange of information on experiences at work (22)(23) . The possibility of expansion of knowledge is greater when there is prior knowledge on the topic (24) .…”

Training mental health promotion workers in programme evaluation

“…2005, Barry 2007). To improve the knowledge base in this area, Ridde et al . (2009) urge workers to present empirical case studies to ascertain what works and what does not work.…”

“…Outside of university settings, there are few documented studies regarding programme evaluation training for workers in community settings. In one such programme, Ridde et al . (2009) describe programme evaluation training for health professionals in francophone Africa, with participants reporting an increase in evaluation mastery, post training.…”

“…However, there have been repeated calls for the expansion and strengthening of its evidence base (WHO 2002, Barry 2007). To improve the knowledge base in this area, Ridde et al (2009) urge workers to present empirical case studies to ascertain what works and what does not work. Programme evaluation is an essential component of developing an evidence base that will assist researchers, workers, policy makers and funders to reach a consensus on the best ways to implement mental health promotion in the future.…”

“…Outside of university settings, there are few documented studies regarding programme evaluation training for workers in community settings. In one such programme, Ridde et al (2009) describe programme evaluation training for health professionals in francophone Africa, with participants reporting an increase in evaluation mastery, post training. However, those who had actually carried out an evaluation 12 months after training reported a negative gap between their declared mastery and their actual practice, suggesting that it is more difficult to implement evaluation skills than it is to understand them.…”