Measuring Coverage in MNCH: A Validation Study Linking Population Survey Derived Coverage to Maternal, Newborn, and Child Health Care Records in Rural China

Abstract

BackgroundAccurate data on coverage of key maternal, newborn, and child health (MNCH) interventions are crucial for monitoring progress toward the Millennium Development Goals 4 and 5. Coverage estimates are primarily obtained from routine population surveys through self-reporting, the validity of which is not well understood. We aimed to examine the validity of the coverage of selected MNCH interventions in Gongcheng County, China.Method and FindingsWe conducted a validation study by comparing women’s self-reported coverage of MNCH interventions relating to antenatal and postnatal care, mode of delivery, and child vaccinations in a community survey with their paper- and electronic-based health care records, treating the health care records as the reference standard. Of 936 women recruited, 914 (97.6%) completed the survey. Results show that self-reported coverage of these interventions had moderate to high sensitivity (0.57 [95% confidence interval (CI): 0.50–0.63] to 0.99 [95% CI: 0.98–1.00]) and low to high specificity (0 to 0.83 [95% CI: 0.80–0.86]). Despite varying overall validity, with the area under the receiver operating characteristic curve (AUC) ranging between 0.49 [95% CI: 0.39–0.57] and 0.90 [95% CI: 0.88–0.92], bias in the coverage estimates at the population level was small to moderate, with the test to actual positive (TAP) ratio ranging between 0.8 and 1.5 for 24 of the 28 indicators examined. Our ability to accurately estimate validity was affected by several caveats associated with the reference standard. Caution should be exercised when generalizing the results to other settings.ConclusionsThe overall validity of self-reported coverage was moderate across selected MNCH indicators. However, at the population level, self-reported coverage appears to have small to moderate degree of bias. Accuracy of the coverage was particularly high for indicators with high recorded coverage or low recorded coverage but high specificity. The study provides insights into the accuracy of self-reports based on a population survey in low- and middle-income countries. Similar studies applying an improved reference standard are warranted in the future.

Can surveys of women accurately track indicators of maternal and newborn care? A validity and reliability study in Kenya

“…Of the four indicators that met both validation criteria in this study, a support person present during birth [3], and cesarean section [4,5] have also been found to be reported accurately by women in prior studies.…”

Measuring Coverage in MNCH: New Findings, New Strategies, and Recommendations for Action

“…A review of DHS and MICS surveys since 2000 found that home-based vaccination records were available for less than 70% of children in 21 of the 33 least developed countries [51]. The report from China in this Collection, which used a combination of home-based records and an electronic system as the reference standard, had high proportions of missing data for all the coverage indicators assessed [24].…”

“…We conducted studies of this type for emergency cesarean sections in Ghana and the Dominican Republic [20], a broad range of interventions delivered around the time of birth (the peripartum period) in Mozambique [21], diagnosis and antibiotic treatment of childhood pneumonia in Pakistan and Bangladesh [22], diagnosis and treatment of malaria in Zambia [23], and selected services across the MNCH continuum of care in a rural population in China [24]. Table 1 presents findings on sensitivity, specificity, and accuracy for a subset of the indicators assessed in the research studies, focusing on global consensus coverage indicators currently in use and indicators closely related to these accepted coverage indicators.…”

“…We believe for the same reason that applying standard cutoff values for acceptable levels of accuracy is not appropriate in this situation. True/actual positives ratios or their mathematical equivalents are presented in the individual papers [20],[21],[23],[24].…”

Measuring Coverage in MNCH: Testing the Validity of Women's Self-Report of Key Maternal and Newborn Health Interventions during the Peripartum Period in Mozambique

“…The six studies from low-resource countries [21]–[26] all validated women's self-reports of obstetric complications comparing follow-up interviews with women who delivered in hospitals against clinical records. In the PLOS Medicine “Measuring Coverage in MNCH” Collection, of which this paper is a part, two other articles address indicators for care in the peripartum period [27], [28].…”