Program evaluation dashboard design and development for Missouri telehealth network show-me echo program

Abstract

Background: The Show-Me ECHO Program is a state funded telehealth project, established in 2014, that connects interdisciplinary teams of experts with rural and isolated primary care providers (PCPs) and other professionals using videoconferencing and interactive case-based learning in an effort to develop advanced skills, best practices and ultimately improve patient care access, quality, and efficiency. Since inception, the Show-Me ECHO program has experienced rapid growth and expansion to over 40 ECHO topics, impacting all 114 Missouri Counties and over 2,300 health/community organizations. The exponential growth experienced by the ECHO model highlights a crucial need for adept program evaluation, reporting tools and resources which will facilitate the process of systematically examining the implementation, quality, impact, and value of the program. Objective: The objective of this project is to design and build data dashboards that support a macro-evaluation and management of Missouri Telehealth Network's Show-Me ECHO program and contributes to program improvement activities. Methods: A stakeholder identification and needs analysis was completed to ensure comprehensive measurement of program performance metrics. Show-Me ECHO program administrative data, clinic information, attendance records for participants and facilitators, case presentation metrics, didactic presentations, and more were extracted from MTN data repositories for the 2014-2021 period and analyzed for dashboard development. Data cleaning and preprocessing was conducted in a combination of Excel, Python and Tableau. The dashboards and other data visualization metrics were created in Tableau. Results: Data extraction generated a total of 70,910 observations across three reports ('Clinic Data', 'Didactic Presentation Data' and 'Patient Presentation Data'). Three preliminary dashboards -- "Show-Me ECHO Project Reach and Attendance" "Show-Me ECHO Project Overview" and "ECHO Clinic Performance Report" were established to provide Missouri Telehealth Network (MTN) teams and stakeholders detailed insight into growth and performance of the Show-Me ECHO project and support development and management of action plans. Conclusions: The constructed MTN Dashboards support organization efforts to establish a single unified approach to monitor program progress, identify and prioritize efforts and resource allocation, identify specific Missouri counties that may benefit from interventions and ECHO clinic expansions, and provide appropriate performance metrics that can be shared with both decision makers and relevant stakeholders. Future considerations for dashboard expansion include incorporating PCP self-efficacy and knowledge surveys and Claims data analysis to enable further tracking of Provider and Patient outcomes. A feasibility assessment of the implementation of dashboards at other superhubs for benchmarking and program outcome comparison studies should also be considered.