MEDHI -Tool - Demystifying the WHO framework For Monitoring and Evaluation of Digital Health Interventions (Preprint)

Abstract

BACKGROUND Digital health interventions (DHI), such as conversational agents and chatbots, play a crucial role for health support, promotion and rehabilitation. As any other health interventions, DHIs require monitoring and evaluation (M&E). However, the current evaluation process lacks a standardized approach and necessitates the use of multiple strategies to ensure comprehensiveness. In 2016, the World Health Organization (WHO) proposed a seven-step practical guide for M&E DHIs, to conducting research and assessment, that became the main and most comprehensive reference in this context. Despite its importance, the guide is complex and lacks decision support tools, making its application a challenging task. OBJECTIVE The aim of this study was to create the MEDHI-Tool to support the use of the WHO M&E guideline for DHIs. METHODS This is applied methodological research, carried out in 2 phases: development and evaluation of the MEDHI-Tool; and application of this tool to M&E, a chatbot for guidance, monitoring and stroke tracking, recently made available to the public and which is being used by people from an association of patients with this disease. The development was carried out in Excel and its content was based on the consolidation of the information present among the chapters of the guide, which were classified into categories (description, objective and recommendations) and annotated in tables, to serve as a basis for creating the filters. The product was evaluated by experts for organization, relevance, understanding and applicability in M&E. RESULTS The MEDHI-Tool has four tabs composed and follow by the sequence of steps of the WHO guide: 1 and 4 (first tab); 2; 3; 5 to 7 (fourth tab). The tool provides a standard of guidelines to be applied to all maturity cycles of any DHI profile. The evaluation of the tool yielded a favorable response regarding each analyzed criterion. In the case study, the MEDHI-Tool proved to be a valuable resource in guiding decisions on appropriate metrics, objectives, study types, data collection, and indicators. CONCLUSIONS It was proposed the development of a tool to guide the M&E of DHI such as chatbots. The MEDHI- Tool has proven to be useful in providing evidence for planning processes, ensuring effective analysis of data generated and improving the delivery of products provided by DHI.