Protocol for Usability Testing and Validation of the ISO Draft International Standard 19223 for Lung Ventilators

Abstract

BackgroundClinicians, such as respiratory therapists and physicians, are often required to set up pieces of medical equipment that use inconsistent terminology. Current lung ventilator terminology that is used by different manufacturers contributes to the risk of usage errors, and in turn the risk of ventilator-associated lung injuries and other conditions. Human factors and communication issues are often associated with ventilator-related sentinel events, and inconsistent ventilator terminology compounds these issues. This paper describes our proposed protocol, which will be implemented at the University of Waterloo, Canada when this project is externally funded.ObjectiveWe propose to determine whether a standardized vocabulary improves the ease of use, safety, and utility as it relates to the usability of medical devices, compared to legacy medical devices from multiple manufacturers, which use different terms.MethodsWe hypothesize that usage errors by clinicians will be lower when standardization is consistently applied by all manufacturers. The proposed study will experimentally examine the impact of standardized nomenclature on performance declines in the use of an unfamiliar ventilator product in clinically relevant scenarios. Participants will be respiratory therapy practitioners and trainees, and we propose studying approximately 60 participants.ResultsThe work reported here is in the proposal phase. Once the protocol is implemented, we will report the results in a follow-up paper.ConclusionsThe proposed study will help us better understand the effects of standardization on medical device usability. The study will also help identify any terms in the International Organization for Standardization (ISO) Draft International Standard (DIS) 19223 that may be associated with recurrent errors. Amendments to the standard will be proposed if recurrent errors are identified. This report contributes a protocol that can be used to assess the effect of standardization in any given domain that involves equipment, multiple manufacturers, inconsistent vocabulary, symbology, audio tones, or patterns in interface navigation. Second, the protocol can be used to experimentally evaluate the ISO DIS 19223 for its effectiveness, as researchers around the world may wish to conduct such tests and compare results.

Findings from the 2018 Yearbook Section on Human Factors and Organizational Issues

“…Although the study group was not representative of typical practitioners but of trainees, the authors provided a very nice methodology that may be reused for those who are interested in analyzing the automation bias. Minotra et al 10 supported a better understanding of the effects of standardization on medical device usability. They proposed a protocol to experimentally examine the impact of standardized nomenclature on performance with the use of an unfamiliar ventilator product.…”

Analysis: Classifying Ventilation Modes and Improving Operator Training with ISO 19223:2019

“…The Centers for Disease Control and Prevention has reported that more than 300,000 patients are ventilated in the United States each year. 1,2 Human factors and communication issues have been cited as the two most common causes of ventilator-related sentinel events, according to reports of patient mortality or severe harm received by The Joint Commission during 2004-15. 2,3 Over a decade ago, it was determined that the scope of the International Electrotechnical Commission (IEC) standard IEC 60601-1:2005 4 had to be extended to consider equipment usability in its third edition.…”

“…1,2 Human factors and communication issues have been cited as the two most common causes of ventilator-related sentinel events, according to reports of patient mortality or severe harm received by The Joint Commission during 2004-15. 2,3 Over a decade ago, it was determined that the scope of the International Electrotechnical Commission (IEC) standard IEC 60601-1:2005 4 had to be extended to consider equipment usability in its third edition. The Joint Working Group (JWG) of the International Organization for Standardization (ISO) and IEC subcommittees responsible for critical care ventilators (ISO/TC 121/SC 3 and IEC TC/SC 62D/JWG 1) concurred that there was a wide recognition that a key factor adversely affecting lung ventilator usability was the lack of a standardized vocabulary relating to the modes of operation for these devices.…”