The usability of ventilators: a comparative evaluation of use safety and user experience

Abstract

BackgroundThe design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.MethodsWe carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator.Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).ResultsNine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen’s d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p = 0.044) and V500 (p = 0.020). Participants reported higher system usability for G5 when compared to PB980 (p = 0.035) and higher system usability for SERVO-U when compared to G5 (p < 0.001), PB980 (p < 0.001), and V500 (p < 0.001). Participants reported lower workload for G5 when compared to PB980 (p < 0.001) and lower workload for SERVO-U when compared to PB980 (p < 0.001) and V500 (p < 0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants’ performance and perception include the low sensitivity of G5’s touchscreen and the positive effect from the quality of SERVO-U’s user interface design.ConclusionsThis study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users’ expectations.Electronic supplementary materialThe online version of this article (doi:10.1186/s13054-016-1431-1) contains supplementary material, which is available to authorized users.

Usability Study of the User-Interface of Intensive Care Ventilators Based on User Test and Eye-Tracking Signals

“…Our usability study of ventilator user-interface showed that most ventilators now used in China have poor ergonomics, with high task error rates (21.2% to 35.0%) and longer task completion times than those reported in published studies [ 17 – 21 ]; this can lead to serious consequences in emergency situations. These can be illustrated by several examples observed in our study.…”

Assessing Use Errors Related to the Interface Design of Electrosurgical Units

“…For example, an HF evaluation of critical care ventilators found that one ventilator model demonstrated higher levels of safety and user experience compared with the other three models evaluated. The results of this evaluation can support ventilator purchasing decisions . Another HF evaluation of two automated chest compression devices found that the interruption in chest compressions when applying either device to the patient was notably longer than the maximum interruption time recommended by the American Heart Association.…”

“…The results of this evaluation can support ventilator purchasing decisions. 7 Another HF evaluation of two automated chest compression devices found that the interruption in chest compressions when applying either device to the patient was notably longer than the maximum interruption time recommended by the American Heart Association. The results of this evaluation prevented the purchase of an automated chest compression device at the authors' health care facility.…”

Protocol for Usability Testing and Validation of the ISO Draft International Standard 19223 for Lung Ventilators

“…The task will mainly involve a pilot run of the main experiment with think-alouds and unstructured interviews. We will recruit approximately 12 experienced respiratory therapists [ 10 ] and instructors from the respiratory therapy program at Conestoga College. The pilot trial and feedback will be used to make modifications in the materials.…”

“…However, the model focuses on the evaluation of medical terminology rather than user interface(s) incorporating a given terminology, and the model does not prescribe a protocol for comparing several systems that incorporate alternative terminologies [ 9 ]. Morita et al [ 10 ] reported a study comparing the safety and user experience of four ventilator models; although their experimental protocol is informative for usability studies involving ventilators, their design is not concerned with nomenclature standardization. Therefore, our protocol to evaluate the usability of a standardized nomenclature applicable across user interfaces of several heterogeneous devices would be a contribution, and the protocol would be applicable in other terminology-related usability studies.…”