Assessment of the validity of the measurement of newborn and maternal health-care coverage in hospitals (EN-BIRTH): an observational study

Abstract

Background Progress in reducing maternal and neonatal deaths and stillbirths is impeded by data gaps, especially regarding coverage and quality of care in hospitals. We aimed to assess the validity of indicators of maternal and newborn health-care coverage around the time of birth in survey data and routine facility register data.Methods Every Newborn-BIRTH Indicators Research Tracking in Hospitals was an observational study in five hospitals in Bangladesh, Nepal, and Tanzania. We included women and their newborn babies who consented on admission to hospital. Exclusion critiera at admission were no fetal heartbeat heard or imminent birth. For coverage of uterotonics to prevent post-partum haemorrhage, early initiation of breastfeeding (within 1 h), neonatal bag-mask ventilation, kangaroo mother care (KMC), and antibiotics for clinically defined neonatal infection (sepsis, pneumonia, or meningitis), we collected time-stamped, direct observation or case note verification data as gold standard. We compared data reported via hospital exit surveys and via hospital registers to the gold standard, pooled using random effects meta-analysis. We calculated population-level validity ratios (measured coverage to observed coverage) plus individual-level validity metrics. Findings We observed 23 471 births and 840 mother-baby KMC pairs, and verified the case notes of 1015 admitted newborn babies regarding antibiotic treatment. Exit-survey-reported coverage for KMC was 99•9% (95% CI 98•3-100) compared with observed coverage of 100% (99•9-100), but exit surveys underestimated coverage for uterotonics (84•7% [79•1-89•5]) vs 99•4% [98•7-99•8] observed), bag-mask ventilation (0•8% [0•4-1•4]) vs 4•4% [1•9-8•1]), and antibiotics for neonatal infection (74•7% [55•3-90•1] vs 96•4% [94•0-98•6] observed). Early breastfeeding coverage was overestimated in exit surveys (53•2% [39•4-66•8) vs 10•9% [3•8-21•0] observed). "Don't know" responses concerning clinical interventions were more common in the exit survey after caesarean birth. Register data underestimated coverage of uterotonics (77•9% [37•8-99•5] vs 99•2% [98•6-99•7] observed), bag-mask ventilation (4•3% [2•1-7•3] vs 5•1% [2•0-9•6] observed), KMC (92•9% [84•2-98•5] vs 100% [99•9-100] observed), and overestimated early breastfeeding (85•9% (58•1-99•6) vs 12•5% [4•6-23•6] observed). Inter-hospital heterogeneity was higher for register-recorded coverage than for exit survey report. Even with the same register design, accuracy varied between hospitals.Interpretation Coverage indicators for newborn and maternal health care in exit surveys had low accuracy for specific clinical interventions, except for self-report of KMC, which had high sensitivity after admission to a KMC ward or corner and could be considered for further assessment. Hospital register design and completion are less standardised than surveys, resulting in variable data quality, with good validity for the best performing sites. Because approximately 80% of births worldwide take place in facilities, standardising register d...

Kangaroo mother care: EN-BIRTH multi-country validation study

“…1). Detailed information regarding the research protocol, methods and analysis have been published separately [18,25]. 1.…”

“…Our study hospitals are all CEmONC hospitals, and the mothers in our sample had higher levels of education than national averages, so our findings might not be generalizable to measurement from KMC provided in other facilities. It is possible that the presence of researchers on the KMC wards/corners could have resulted in improved care or register documentation by health workers [25]. In Pokhara, the inter-rater reliability agreement for observation were unexpectedly low and might have affected validation results in that site.…”

Counting on birth registration: mixed-methods research in two EN-BIRTH study hospitals in Tanzania

“…This birth registration study uses primary data collected during the EN-BIRTH study and the detailed study research protocol and validation results have been published separately [20,21].…”

Barriers and enablers to routine register data collection for newborns and mothers: EN-BIRTH multi-country validation study

“…Indicators were selected based on criteria outlined in global frameworks [6,28,29]. The EN-BIRTH validation assessment reported finding register-recorded coverage accuracy varied by indicator and by hospital [30].…”