Assessing the validity of prospective hazard analysis methods: a comparison of two techniques

Abstract

BackgroundProspective Hazard Analysis techniques such as Healthcare Failure Modes and Effects Analysis (HFMEA) and Structured What If Technique (SWIFT) have the potential to increase safety by identifying risks before an adverse event occurs. Published accounts of their application in healthcare have identified benefits, but the reliability of some methods has been found to be low. The aim of this study was to examine the validity of SWIFT and HFMEA by comparing their outputs in the process of risk assessment, and comparing the results with risks identified by retrospective methods.MethodsThe setting was a community-based anticoagulation clinic, in which risk assessment activities had been previously performed and were available. A SWIFT and an HFMEA workshop were conducted consecutively on the same day by experienced experts. Participants were a mixture of pharmacists, administrative staff and software developers. Both methods produced lists of risks scored according to the method’s procedure. Participants’ views about the value of the workshops were elicited with a questionnaire.ResultsSWIFT identified 61 risks and HFMEA identified 72 risks. For both methods less than half the hazards were identified by the other method. There was also little overlap between the results of the workshops and risks identified by prior root cause analysis, staff interviews or clinical governance board discussions. Participants’ feedback indicated that the workshops were viewed as useful.ConclusionsAlthough there was limited overlap, both methods raised important hazards. Scoping the problem area had a considerable influence on the outputs. The opportunity for teams to discuss their work from a risk perspective is valuable, but these methods cannot be relied upon in isolation to provide a comprehensive description. Multiple methods for identifying hazards should be used and data from different sources should be integrated to give a comprehensive view of risk in a system.

An analysis of electronic health record–related patient safety incidents

“…Different patient safety data sources that complement each other are useful in identifying hazards and providing a more comprehensive view of the risks in a particular system. 23,24 Adverse events occurring in one institution are known to recur in other institutions, often with the same causes and contributing factors. 25 By identifying the nature of patient safety incidents, initiatives for improvement can be developed and prioritised.…”

Building Evidence the Federal Aviation Administration's UAS Safety Strategy Needs Improvement

“…Differences were also found between conditions which used consensuses to determine an overall severity rating and those that merely averaged individual ratings (Ashley & Armitage, 2010). Potts et al (2014) investigated the validity of structured risk analysis methods. They were specifically interested in comparing the resultant outcomes of two conditions: analysis using different techniques, and if different groups can replicate risk analysis results using the same technique.…”

“…They were specifically interested in comparing the resultant outcomes of two conditions: analysis using different techniques, and if different groups can replicate risk analysis results using the same technique. They investigated the Structured What If Technique (SWIFT) and HFMEA (FMEA tailored to the healthcare industry) in a workshop setting (Potts et al, 2014). Teams of five participants per group worked together to make decisions using each of the two techniques.…”

“…In turn, HFMEA had 42 (58.3 percent) risks that were not identified by SWIFT. Additionally, the researchers concluded that there was little overlap of high risks items between the two analyses techniques (Potts et al, 2014).…”

Reverse Evaluation of Failure Mode and Effects Analysis Model: A New Reliable Performance Measurement

“…A control experiment study investigated the reliability of FMEA, shows that a significant difference in severity verdict in each group, to conclude that the FMEA is not a reliable tool [13]. Another drawback is the outcome inconsistency in the outcome of the FMEA [14].…”