Impact of a district-wide health center strengthening intervention on healthcare utilization in rural Rwanda: Use of interrupted time series analysis

Abstract

BackgroundEvaluations of health systems strengthening (HSS) interventions using observational data are rarely used for causal inference due to limited data availability. Routinely collected national data allow use of quasi-experimental designs such as interrupted time series (ITS). Rwanda has invested in a robust electronic health management information system (HMIS) that captures monthly healthcare utilization data. We used ITS to evaluate impact of an HSS intervention to improve primary health care facility readiness on health service utilization in two rural districts of Rwanda.MethodsWe used controlled ITS analysis to compare changes in healthcare utilization at health centers (HC) that received the intervention (n = 13) to propensity score matched non-intervention health centers in Rwanda (n = 86) from January 2008 to December 2012. HC support included infrastructure renovation, salary support, medical equipment, referral network strengthening, and clinical training. Baseline quarterly mean outpatient visit rates and population density were used to model propensity scores. The intervention began in May 2010 and was implemented over a twelve-month period. We used monthly healthcare utilization data from the national Rwandan HMIS to study changes in the (1) number of facility deliveries per 10,000 women, (2) number of referrals for high risk pregnancy per 100,000 women, and (3) the number of outpatient visits performed per 1,000 catchment population.ResultsPHIT HC experienced significantly higher monthly delivery rates post-HSS during the April-June season than comparison (3.19/10,000, 95% CI: [0.27, 6.10]). In 2010, this represented a 13% relative increase, and in 2011, this represented a 23% relative increase. The post-HSS change in monthly rate of high-risk pregnancies referred increased slightly in intervention compared to control HC (0.03/10,000, 95% CI: [-0.007, 0.06]). There was a small immediate post-HSS increase in outpatient visit rates in intervention compared to control HC (6.64/1,000, 95% CI: [-13.52, 26.81]).ConclusionWe failed to find strong evidence of post-HSS increases in outpatient visit rates or referral rates at health centers, which could be explained by small sample size and high baseline nation-wide health service coverage. However, our findings demonstrate that high quality routinely collected health facility data combined with ITS can be used for rigorous policy evaluation in resource-limited settings.

Using routine health information data for research in low- and middle-income countries: a systematic review

“…RHIS data have been used to evaluate a wide range of interventions, ranging from programs that targeted specific diseases to interventions or policies that affected multiple types of diseases or health services. These included: the effect of malaria control strategies [30][31][32][33][34][35][36], user fee exemption policies [37][38][39][40], health financing schemes [41][42][43][44], interventions on health governance [45][46][47][48][49][50][51][52][53], the administration of new vaccines and vaccination campaigns [54][55][56], as well as community-level interventions such as approaches to enhance community participation and improve referrals from traditional birth attendants in increasing the demand for maternal and child care [57][58][59].…”

“…Nonetheless, ITS analyses can be affected by time-varying confounders that rapidly change and some models included contextual factors from other data sources, such as climate and program data. To strengthen the quasi-experimental design, two studies also included a contrast group of time series to control for contextual changes that occurred at the same time as the interventions [38,45].…”

“…These strategies consisted of exclusion, imputation, interpolation, verification, and accounting for missing data in modeling. Exclusion of missing data was the most common practice, and among studies that used this technique, they excluded facilities from the analytic samples [38,41,45,52,65,79,83,84,87,94,96,121], restricted the study period based on explicit criteria [54,122], or applied sensitivity analysis to compare various exclusion criteria [41,89,90]. Imputation methods varied from assigning specific values to the missing observation [42,87,118,[123][124][125], to various modeling strategies such as conditional autoregressive model [110], generalized linear regression [124], and iterative singular value decomposition [124].…”

Using Routine Health Information Data for Research in Low- and Middle-Income Countries: A Systematic Review

“…RHIS data have been used to evaluate a wide range of interventions, ranging from programs that targeted speci c diseases to interventions or policies that affected multiple types of diseases or health services. These included: the effect of malaria control strategies [30][31][32][33][34][35][36] , user fee exemption policies [37][38][39][40] , health nancing schemes [41][42][43][44] , interventions on health governance [45][46][47][48][49][50][51][52][53] , the administration of new vaccines and vaccination campaigns [54][55][56] , as well as community-level interventions such as approaches to enhance community participation and improve referrals from traditional birth attendants in increasing the demand for maternal and child care [57][58][59] .…”

“…These strategies consisted of exclusion, imputation, interpolation, veri cation, and accounting for missing data in modeling. Exclusion of missing data was the most common practice, and among studies that used this technique, they excluded facilities from the analytic samples 38,41,45,52,65,91,[93][94][95][96][97][98] , restricted the study period based on explicit criteria 54,99 , or applied sensitivity analysis to compare various exclusion criteria 41,100,101 . Imputation methods varied from assigning speci c values to the missing observation 42,86,91,[102][103][104] , to various modeling strategies such as conditional autoregressive model 87 , generalized linear regression 103 , and iterative singular value decomposition 103 .…”