Measuring Coverage in MNCH: A Prospective Validation Study in Pakistan and Bangladesh on Measuring Correct Treatment of Childhood Pneumonia

Abstract

BackgroundAntibiotic treatment for pneumonia as measured by Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS) is a key indicator for tracking progress in achieving Millennium Development Goal 4. Concerns about the validity of this indicator led us to perform an evaluation in urban and rural settings in Pakistan and Bangladesh.Methods and FindingsCaregivers of 950 children under 5 y with pneumonia and 980 with “no pneumonia” were identified in urban and rural settings and allocated for DHS/MICS questions 2 or 4 wk later. Study physicians assigned a diagnosis of pneumonia as reference standard; the predictive ability of DHS/MICS questions and additional measurement tools to identify pneumonia versus non-pneumonia cases was evaluated.Results at both sites showed suboptimal discriminative power, with no difference between 2- or 4-wk recall. Individual patterns of sensitivity and specificity varied substantially across study sites (sensitivity 66.9% and 45.5%, and specificity 68.8% and 69.5%, for DHS in Pakistan and Bangladesh, respectively). Prescribed antibiotics for pneumonia were correctly recalled by about two-thirds of caregivers using DHS questions, increasing to 72% and 82% in Pakistan and Bangladesh, respectively, using a drug chart and detailed enquiry.ConclusionsMonitoring antibiotic treatment of pneumonia is essential for national and global programs. Current (DHS/MICS questions) and proposed new (video and pneumonia score) methods of identifying pneumonia based on maternal recall discriminate poorly between pneumonia and children with cough. Furthermore, these methods have a low yield to identify children who have true pneumonia. Reported antibiotic treatment rates among these children are therefore not a valid proxy indicator of pneumonia treatment rates. These results have important implications for program monitoring and suggest that data in its current format from DHS/MICS surveys should not be used for the purpose of monitoring antibiotic treatment rates in children with pneumonia at the present time. Please see later in the article for the Editors' Summary

Care seeking behaviour and aspects of quality of care by caregivers for children under five with and without pneumonia in Ibadan, Nigeria

“…DHS and MICS are not designed to specifically identify cases of pneumonia and they label these episodes as ‘symptoms of acute respiratory infection (ARI)’. One prospective study in Pakistan and Bangladesh also found current methods discriminate poorly between children with pneumonia and those with cough/cold resulting in a poor yield of those who have pneumonia [13]. Therefore, a large proportion of the data gathered for care seeking from these surveys actually reflect the behaviours of those seeking care for children with symptoms of ARI (mostly comprising children with cough/cold).…”

“…However, previous evidence suggests that these surveys do not discriminate well between cases of cough/cold with difficult breathing and pneumonia [13]. This study therefore also aimed to investigate the differences in reported care seeking behaviour and treatment for children with and without pneumonia using these survey tools.…”

Measuring Coverage in MNCH: Tracking Progress in Health for Women and Children Using DHS and MICS Household Surveys

“…For example, a parent cannot know a child's birthweight if the child was not weighed at birth and is unlikely to remember the exact timing of tetanus toxoid vaccinations received over a lifetime. Decisions about what questions respondents can answer with acceptable accuracy are generally made by survey designers in consultation with technical experts; the research papers in this collection provide some of the first rigorous research assessing the validity of self-reported exposure to RMNCH interventions [17],[18],[25]–[28]. For coverage questions that cannot be answered through household surveys, alternative methods should be explored.…”

Measuring Coverage in MNCH: Challenges and Opportunities in the Selection of Coverage Indicators for Global Monitoring

“…A good example of compromise is the inclusion by both Countdown and the Commission of an indicator on coverage for antibiotic treatment of childhood pneumonia. Pneumonia, the number one killer of children worldwide [27] can be treated effectively and at relatively low cost with a course of antibiotics [28] but, as reported elsewhere in this Collection [20],[21] and summarized in Table S1, the current global coverage indicator for treatment does not produce accurate results. Its measurement limitations suggest that additional related indicators may need to be collected until data collection methods are improved (i.e., care seeking for pneumonia).…”