User Interface Problems of a Nationwide Inpatient Information System: A Heuristic Evaluation

Abstract

SummaryIntroduction: While studies have shown that usability evaluation could uncover many design problems of health information systems, the usability of health information systems in developing countries using their native language is poorly studied. The objective of this study was to evaluate the usability of a nationwide inpatient information system used in many academic hospitals in Iran. Material and Methods: Three trained usability evaluators independently evaluated the system using Nielsen's 10 usability heuristics. The evaluators combined identified problems in a single list and independently rated the severity of the problems. We statistically compared the number and severity of problems identified by HIS experienced and non-experienced evaluators. Results: A total of 158 usability problems were identified. After removing duplications 99 unique problems were left. The highest mismatch with usability principles was related to "Consistency and standards" heuristic (25%) and the lowest related to "Flexibility and efficiency of use" (4%). The average severity of problems ranged from 2.4 (Major problem) to 3.3 (Catastrophe problem). The experienced evaluator with HIS identified significantly more problems and gave higher severities to problems (p<0.02). Discussion: Heuristic Evaluation identified a high number of usability problems in a widely used inpatient information system in many academic hospitals. These problems, if remain unsolved, may waste users' and patients' time, increase errors and finally threaten patient's safety. Many of them can be fixed with simple redesign solutions such as using clear labels and better layouts. This study suggests conducting further studies to confirm the findings concerning effect of evaluator experience on the results of Heuristic Evaluation.

Usability Evaluation of Three Admission and Medical Records Subsystems Integrated into Nationwide Hospital Information Systems: Heuristic Evaluation

“…The help heuristic was placed in the group of “highest rate of problems” in three evaluated HISs. This result is consistent with the results of two previous studies ( 24 , 27 ). This finding shows that HIS designers do not pay necessary attention to help and documentation and may consider it as an unnecessary and secondary functionality ( 28 ).…”

“…The checklist can regulate the heuristic evaluation process and avoid subjective comments about user interface problems. Secondly, although the results of this study on the HISs problems may not be generalizable to other health information systems, these results are consistent with the results of similar studies performed on HISs in many cases ( 15 , 16 , 24 , 27 ). This study also has two limitations that warrant consideration.…”

“…This heuristic relates to the use of shortcuts (such as functional keys of F1-F12) for frequently used commands to accelerate user interaction with the system. Similarly, some other studies have shown that this heuristic has fewer problems than the other heuristics ( 16 , 24 ). The low rate of problems associated with this heuristic may have been due to the evaluators expertise in interaction with HISs.…”

Evaluating the Usability of a Nationwide Pharmacy Information System in Iran: Application of Nielson’s Heuristics

“…Our results indicated that the heuristics "consistency and standard" and "recognition rather than recall" have the largest number of violations, with the lowest number of violations related to "error prevention" and "Help users recognize, diagnose, and recover from errors" heuristics. These results are similar to the results of other studies, such as the those conducted by Khajouei et al, Atashi et al, and Nabovati et al in Iran (25)(26)(27), and also the studies conducted by Choi and Bakken, Georgsson et al, Joshi et al, and Mirkovic et al in countries other than Iran, which reported a high number of violations concerning these heuristics (28)(29)(30)(31). On the other hand, the unexpected results of this study indicated that the severity of 60.8% of all violations is major and catastrophic, which accounts for more than half of the identified violations and, in this case, and efficiency of the systems, satisfaction of their users, and hospital's financial issues.…”

Determinants of user acceptance of a specific social platform for older adults: An empirical examination of user interface characteristics and behavioral intention

“…It was put forward by Nielsen, and after a period of development, ten usability heuristics were formally proposed in 1995 [ 24 ]. Currently, heuristic evaluation checklists developed in studies on system usability evaluation are based on the ten heuristics proposed by Nielsen as evaluation items [ 25 – 27 ]. Based on Nielsen’s ten usability heuristics and Shneiderman and Plaisant’s eight Golden Rules of Interface Design [ 28 ], Zhang et al [ 29 ] proposed 14 heuristics, adding items such as “informative feedback,” “clear closure,” “reversible actions” and “use user's language.” The four additional items were originally applied to determine the safety and usability of the 1-channel volumetric infusion pump used by patients.…”