Unmet Need: Improving mHealth Evaluation Rigor to Build the Evidence Base

Abstract

mHealth-the use of mobile technologies for health-is a growing element of health system activity globally, but evaluation of those activities remains quite scant, and remains an important knowledge gap for advancing mHealth activities. In 2010, the World Health Organization and Columbia University implemented a small-scale survey to generate preliminary data on evaluation activities used by mHealth initiatives. The authors describe self-reported data from 69 projects in 29 countries. The majority (74%) reported some sort of evaluation activity, primarily nonexperimental in design (62%). The authors developed a 6-point scale of evaluation rigor comprising information on use of comparison groups, sample size calculation, data collection timing, and randomization. The mean score was low (2.4); half (47%) were conducting evaluations with a minimum threshold (4+) of rigor, indicating use of a comparison group, while less than 20% had randomized the mHealth intervention. The authors were unable to assess whether the rigor score was appropriate for the type of mHealth activity being evaluated. What was clear was that although most data came from mHealth projects pilots aimed for scale-up, few had designed evaluations that would support crucial decisions on whether to scale up and how. Whether the mHealth activity is a strategy to improve health or a tool for achieving intermediate outcomes that should lead to better health, mHealth evaluations must be improved to generate robust evidence for cost-effectiveness assessment and to allow for accurate identification of the contribution of mHealth initiatives to health systems strengthening and the impact on actual health outcomes.

Development, roll-out and impact of Xpert MTB/RIF for tuberculosis: what lessons have we learnt and how can we do better?

“…eHealth solutions can serve as a catalyst to overcome specific constraints and can help increase the impact of a known intervention. However, to date, experience with implementation of eHealth solutions has been fragmented and variable, with a lack of data demonstrating impact on healthcare delivery and patient outcomes [ 15 , 16 ]. Connectivity within diagnostic platforms has become an evident pre-requisite [ 17 ].…”

Rifampicin resistant tuberculosis in presumptive pulmonary tuberculosis cases in Dubti Hospital, Afar, Ethiopia

“…Since more than 90% RIF resistant strains are also resistant to INH [7][8][9][10], determining the RIF resistant strains, using the gene X-pert assay a rapid and sensitive technique, is an alternative approach to detect MDR-TB [11,9,10], and this could be explained by rate of spontaneous mutation rate; in which INH target gene has 100 times higher rate of mutation than that of RIF target gene [12]. Therefore, in high MDR-TB burden countries, diagnosis of RIF resistant TB is regarded as MDR-TB [9].…”

“…The very high burden of RIF resistant TB in Ethiopia could be attributed to four broad influential classes of risk factors: Limited coverage of rapid laboratory diagnosis for RIF resistant MTB, improper treatment of drug-susceptible TB, contact with MDR-TB patients and low socioeconomic status [12][13][14]. History of previous TB treatment, history of TB [15][16][17][18][19], monthly income < 500 Ethiopian Birr and being urban resident [15], being male and contact with the known MDR TB patient [20] have been reported as significant risk factors for the emergence of MDR TB in Ethiopia [10,12].…”

Qualitative Assessment of the Feasibility, Usability, and Acceptability of a Mobile Client Data App for Community-Based Maternal, Neonatal, and Child Care in Rural Ghana

“…User-centered design and robust needs assessments are recognized as best practices for development technology [34]. Equipping frontline health workers with mobile phone applications may enhance the delivery of maternal, neonatal, and child health services as well as the accuracy of data capture and recording [16, 26, 35, 36]. Yet, little research exists on health worker perceptions of mobile applications intended to improve maternal, neonatal, and child data management, despite user views being critical to product uptake and implementation [37–39].…”