Measuring Adverse Events in Helicopter Emergency Medical Services: Establishing Content Validity

Abstract

Introduction We sought to create a valid framework for detecting Adverse Events (AEs) in the high-risk setting of Helicopter Emergency Medical Services (HEMS). Methods We assembled a panel of 10 expert clinicians (n=6 emergency medicine physicians and n=4 prehospital nurses and flight paramedics) affiliated with a large multi-state HEMS organization in the Northeast U.S. We used a modified Delphi technique to develop a framework for detecting AEs associated with the treatment of critically ill or injured patients. We used a widely applied measure, the Content Validity Index (CVI), to quantify the validity of the framework’s content. Results The expert panel of 10 clinicians reached consensus on a common AE definition and four-step protocol/process for AE detection in HEMS. The consensus-based framework is composed of three main components: 1) a trigger tool, 2) a method for rating proximal cause, and 3) a method for rating AE severity. The CVI findings isolate components of the framework considered content valid. Conclusions We demonstrate a standardized process for the development of a content valid framework for AE detection. The framework is a model for the development of a method for AE identification in other settings, including ground-based EMS.

Simulation-based training and assessment of non-technical skills in the Norwegian Helicopter Emergency Medical Services: a cross-sectional survey

“…The provision of ALS to critically ill and injured patients in helicopter emergency medical services (HEMS) is a complex process characterised by shifting workload and goals, ill-structured problems, uncertainty, intense time pressure, high stakes and a set of individually complex and interacting tasks of flight-operative, medical, technical, rescue and multidisciplinary character. 1 This process is prone to human error, adverse events and ultimately iatrogenic injury, 2 3 which are to a large degree preventable. 1 4 5 The Norwegian HEMS conduct more than 7500 urgent and interhospital air medical patient transfers annually.…”

Development of a trigger tool to identify adverse events and harm in Emergency Medical Services

“…The Medline search revealed a single peer-reviewed paper that applied a TT-like approach within the EMS setting 21. While rigorously and extensively developed, the tool was limited to a small subset of EMS: Air Medical Services.…”

A Comparative Assessment of Adverse Event Classification in the Out-of-hospital Setting

“…8 It includes a trigger tool and multi-step process for reviewing a medical record that is similar in overall structure to other processes used for medical record review, including those used in the Harvard Medical Practice studies. 9 Use of these frameworks and a common structure promotes consistency, use of a common taxonomy, and improves the possibility of comparing results between different organizations/settings.…”

“…We have developed a content valid framework for AE identification using medical record review that incorporates an EMS-specific AE definition, use of a trigger tool, and a procedure for rating outcome categories of AEs. 8 In this paper, we evaluate this tool by assessing reliability (agreement) with a criterion standard. We hypothesized that an approach to medical record review using consensus/open-discussion would result in higher reliability than reviews utilizing traditional independent clinician review.…”