Monitoring and Evaluation of National Vaccination Implementation: A Scoping Review of How Frameworks and Indicators Are Used in the Public Health Literature

Abstract

An effective Monitoring and Evaluation (M&E) framework helps vaccination programme managers determine progress and effectiveness for agreed indicators against clear benchmarks and targets. We aimed to identify the literature on M&E frameworks and indicators used in national vaccination programmes and synthesise approaches and lessons to inform development of future frameworks. We conducted a scoping review using Arksey and O’Malley’s six-stage framework to identify and synthesise sources on monitoring or evaluation of national vaccination implementation that described a framework or indicators. The findings were summarised thematically. We included 43 eligible sources of 4291 screened. Most (95%) were in English and discussed high-income (51%) or middle-income (30%) settings, with 13 in Europe (30%), 10 in Asia-Pacific (23%), nine in Africa (21%), and eight in the Americas (19%), respectively, while three crossed regions. Only five (12%) specified the use of an M&E framework. Most (32/43; 74%) explicitly or implicitly included vaccine coverage indicators, followed by 12 including operational (28%), five including clinical (12%), and two including cost indicators (5%). The use of M&E frameworks was seldom explicit or clearly defined in our sources, with indicators rarely fully defined or benchmarked against targets. Sources focused on ways to improve vaccination programmes without explicitly considering ways to improve assessment. Literature on M&E framework and indicator use in national vaccination programmes is limited and focused on routine childhood vaccination. Therefore, documentation of more experiences and lessons is needed to better inform vaccination M&E beyond childhood.

Establishing a Framework for Evaluating the Effectiveness of Vaccines Targeting National Vaccination Programs

“…An effective VE evaluation framework is crucial for vaccination program managers to track progress and effectiveness against benchmarks; however, literature on monitoring frameworks and indicators used in national vaccination programs is limited and primarily focuses on routine childhood vaccination, highlighting the need for further documentation and analysis to inform future efforts. 11 …”

First Year of Special Issue “New Insights in Vaccination and Public Health”: Opinions and Considerations

“…Several topics were addressed in the Special Issue “New Insights in Vaccination and Public Health” in its first year of publication. Many of these explored the phenomenon of vaccine hesitancy [ 17 , 20 , 23 , 25 , 26 , 27 , 28 , 30 , 31 , 32 ] and, in particular, the acceptability of the COVID-19 vaccine [ 17 , 25 , 26 , 30 , 31 ] and the anti-influenza vaccine [ 23 , 28 , 33 ]. The role of ICTs was examined in relation to their influence on health choices [ 21 , 23 , 30 ] and in relation to the use of new technologies in the field of vaccinology [ 19 ].…”

Pharmacovigilance in Vaccines: Importance, Main Aspects, Perspectives, and Challenges—A Narrative Review

“…The primary goal of such monitoring is to identify and assess adverse events linked to drugs, medical devices, vaccines, and other health products regulated by health authorities [ 90 ]. As an integral part of the evaluation process for vaccine safety and efficacy, monitoring adverse reactions plays a pivotal role in leveraging this information to protect public health [ 27 , 158 , 159 ].…”

“…These population-based registers enable the assessment of dynamic vaccination programs, monitoring of time trends, and identification of sub-populations with low vaccination coverage [ 180 ]. Continuous monitoring of the quality of these registers is crucial for ensuring reliable evaluation of vaccine uptake and impact at a national level [ 27 , 181 ]. Additionally, the documentation generated through this process can inform improvements in the design and quality of other vaccination or healthcare registers, contributing to more effective public health interventions [ 180 ].…”

“…Experts believe that underreporting of adverse events occurs in all countries because individuals, rather than an automated case detection system, perform reporting [ 25 ]. Estimating underreporting by competent bodies, with values incorporated into the evaluation and monitoring of immunobiologicals through frequent updates, is indispensable [ 26 , 27 ]. Thus, the aim of this study is to explore the significance of pharmacovigilance in relation to vaccines, especially virus; examine the key aspects involved in monitoring the safety and efficacy of vaccines; and discuss future perspectives and challenges in this critical area of public health.…”