Assessing the validity of indicators of the quality of maternal and newborn health care in Kenya

Abstract

BackgroundThe measurement of progress in maternal and newborn health often relies on data provided by women in surveys on the quality of care they received. The majority of these indicators, however, including the widely tracked “skilled attendance at birth” indicator, have not been validated. We assess the validity of a large set of maternal and newborn health indicators that are included or have the potential to be included in population–based surveys.MethodsWe compare women’s reports of care received during labor and delivery in two Kenyan hospitals prior to discharge against a reference standard of direct observations by a trained third party (n = 662). We assessed individual–level reporting accuracy by quantifying the area under the receiver operating curve (AUC) and estimated population–level accuracy using the inflation factor (IF) for each indicator with sufficient numbers for analysis.FindingsFour of 41 indicators performed well on both validation criteria (AUC>0.70 and 0.75<IF<1.25). These were: main provider during delivery was a nurse/midwife, a support companion was present at birth, cesarean operation, and low birthweight infant (<2500 g). Twenty–one indicators met acceptable levels for one criterion only (11 for AUC; 9 for IF). The skilled birth attendance indicator met the IF criterion only.InterpretationFew indicators met both validation criteria, partly because many routine care interventions almost always occurred, and there was insufficient variation for robust analysis. Validity is influenced by whether the woman had a cesarean section, and by question wording. Low validity is associated with indicators related to the timing or sequence of events. The validity of maternal and newborn quality of care indicators should be assessed in a range of settings to refine these findings.

“Every Newborn-BIRTH” protocol: observational study validating indicators for coverage and quality of maternal and newborn health care in Bangladesh, Nepal and Tanzania

“…EN-BIRTH is the first large study to assess validity of newborn and maternal care indicators in routine data systems, doing so at very large scale (>20 000 observed births) across three countries with a high-burden of mortality. Previous maternal and newborn indicator validation studies have focused on testing the validity of women' s self-report method, used in population-based household surveys [25,[46][47][48]64].…”

“…Comparison of the reported indicator to an external data source "gold standard" is recommended [45]. Previous validation studies have mainly focused on population-based intervention coverage indicators for use in household surveys [25,[46][47][48]. Observational studies to determine accuracy of facility registers in high burden settings have typically focused on outcome indicators [29].…”

“…Specificity of routine data will be assessed for those indicators with true negatives and confidence intervals will be computed. "Area Under the Curve" previously used for coverage indicators validation will be used for indicators with true negatives [25,[46][47][48]64].…”

Childbirth care in Egypt: a repeat cross-sectional analysis using Demographic and Health Surveys between 1995 and 2014 examining use of care, provider mix and immediate postpartum care content

“…Given the systemic issues influencing quality of care in Egypt [58] and documented substandard intrapartum practices [34,35], tools that can capture better measures of quality of care during and after delivery should be developed. However, such data and indicators are unlikely to be collected from women themselves, due to poor validity [64][65][66][67]. Finally, it was not possible to assess whether women in public hospitals were in a public or a paying/private section.…”

Discordance in postnatal care between mothers and newborns: Measurement artifact or missed opportunity?

“…Measurement errors in PNC contact also come from the recall interviews with mothers. Several studies have shown recall issues from mothers on maternal and newborn health care questions [9][10][11]. A qualitative study among women in Ghana revealed that mothers can only recall checks that were easily observed and for which the health provider explained the services being provided to the mothers.…”