Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises

Abstract

Results of this project show that users found the newly developed toolkit to be user friendly and more relevant to measurement of specific public health and health care capabilities than other tools currently available. The developed toolkit may contribute to the further advancement of developing a valid approach to exercise performance measurement.

Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

“…Agboola et al developed an online toolkit for measuring performance in emergency response exercises using performance measures tested in over 60 emergency preparedness exercises 15 . The toolkit was pilot tested in collaboration with 10 public health agencies and 4 health care agencies from 8 states.…”

“…However, a few of the tools mentioned in the manuscript are available online (Table 1), including Emergency Preparedness Exercise Evaluation Toolkit, Evaluation Toolkit for the Deployment of MRC Units during Flu Clinics and other Public Health Activities, and Assessment for Disaster Engagement with Partners Toolkit. 15,16,34 A future research and evaluation program for PHEPR system measurement must be informed by a sound conceptual model of how factors in this complex system interact to influence the course of response and subsequent recovery. Lacking such a model, the field may continue to struggle to find consensus on what is most important to measure and, in turn, may continue to produce metrics that are difficult to apply and validate across different contexts.…”

“…Most of these metrics have seen limited dissemination, use, and evaluation outside the research study setting. However, a few of the tools mentioned in the manuscript are available online (Table 1), including Emergency Preparedness Exercise Evaluation Toolkit, Evaluation Toolkit for the Deployment of MRC Units during Flu Clinics and other Public Health Activities, and Assessment for Disaster Engagement with Partners Toolkit 15 16 , 34 A future research and evaluation program for PHEPR system measurement must be informed by a sound conceptual model of how factors in this complex system interact to influence the course of response and subsequent recovery.…”

“…The investigators suggest that this internet-based tool can be used to identify and rank gaps in emergency preparedness capacity and capability at the local level across a range of emergency scenarios. 15 The toolkit was pilot tested in collaboration with 10 public health agencies and 4 health care agencies from 8 states. Thirteen (93%) exercise planners reported that the performance measures generated from the toolkit were appropriate for the creation of exercise evaluation forms to collect data useful for evaluating their organization's performance during the exercise and preferred the toolkit to the Homeland Security Exercise Evaluation Program exercise evaluation guides.…”

Setting Foundations for Developing Disaster Response Metrics

“…7 Toolkits have been developed to address this but are often tailored for measuring performance in response exercises or infectious disease outbreaks. 7,8 Without performance measures commonly agreed upon and used, responders risk repeating mistakes of the past: limited coordination, suboptimal utilization of aid, and limited access to lifesaving care. 9 Review of past disaster responses can inform the construction of performance measures that ultimately will improve disaster relief.…”