An integrated hospital-district performance evaluation for communicable diseases in low-and middle-income countries: Evidence from a pilot in three sub-Saharan countries

Abstract

Introduction The last two decades saw an extensive effort to design, develop and implement integrated and multidimensional healthcare evaluation systems in high-income countries. However, in low- and middle-income countries, few experiences of such systems implementation have been reported in the scientific literature. We developed and piloted an innovative evaluation tool to assess the performance of health services provision for communicable diseases in three sub-Saharan African countries. Material and methods A total of 42 indicators, 14 per each communicable disease care pathway, were developed. A sub-set of 23 indicators was included in the evaluation process. The communicable diseases care pathways were developed for Tuberculosis, Gastroenteritis, and HIV/AIDS, including indicators grouped in four care phases: prevention (or screening), diagnosis, treatment, and outcome. All indicators were calculated for the period 2017–2019, while performance evaluation was performed for the year 2019. The analysis involved four health districts and their relative hospitals in Ethiopia, Tanzania, and Uganda. Results Substantial variability was observed over time and across the four different districts. In the Tuberculosis pathway, the majority of indicators scored below the standards and below-average performance was mainly reported for prevention and diagnosis phases. Along the Gastroenteritis pathway, excellent performance was instead evaluated for most indicators and the highest scores were reported in prevention and treatment phases. The HIV/AIDS pathway indicators related to screening and outcome phases were below the average score, while good or excellent performance was registered within the treatment phase. Conclusions The bottom-up approach and stakeholders’ engagement increased local ownership of the process and the likelihood that findings will inform health services performance and quality of care. Despite the intrinsic limitations of data sources, this framework may contribute to promoting good governance, performance evaluation, outcomes measurement and accountability in settings characterised by multiple healthcare service providers.

Evaluating Healthcare Performance in Low- and Middle-Income Countries: A Pilot Study on Selected Settings in Ethiopia, Tanzania, and Uganda

“…The indicators were calculated both at hospital and district level. Hospital-level indicators were extracted from the registers of hospitals' departments, whilst district-level indicators were retrieved from the District Health Information System (DHIS) of Ethiopia, Uganda, and Tanzania [55]. With reference to hospital registers, the hospitals of Wolisso and Matany had electronic information systems, whereas Tosamaganga and Aber hospitals had paper-based information systems.…”

“…With respect to the final version of the list of defined indicators, some of the indicators were evaluated through the process designed and implemented by the MeS Lab research group in Italy, as inspired by the IRPES [20]. The indicators were evaluated through the identification of five bands, considering the statistical distribution of the indicators' values [20,54,55]. Evaluation scores were built using an algorithm matching each band with a value between 0 and 5, and a colour spanning from red (a very low score) to dark green (a very high score) [54,55].…”

“…The indicators were evaluated through the identification of five bands, considering the statistical distribution of the indicators' values [20,54,55]. Evaluation scores were built using an algorithm matching each band with a value between 0 and 5, and a colour spanning from red (a very low score) to dark green (a very high score) [54,55]. The band's construction varies according to the sign of the indicator, which can either show signs of increasing or decreasing (Activity 4, Table 2).…”

“…The evaluation was performed for the year 2020 only. The evaluation scores were determined by either using international standards identified in the literature when available, or using the benchmarking assessment of values statistical distribution, as described in Tavoschi et al [55]. The RT evaluated each indicator by taking into consideration if the identified reference standard could be adopted across all hospitals and health districts included in the study.…”

“…In order to overcome a static representation and provide an integrated and continuous view of the performance across different settings [33], we also considered the entire patient journey along different care pathways, metaphorically represented by the music stave (the "stave"). The stave illustrates the patients' care pathway, allowing the user to focus on the strengths and weaknesses of the healthcare services delivered to patients along the continuum of care [55]. The previously described process also applies to care pathways, for which the stave, similarly to the dartboard, uses five colour bands (from red to dark green).…”