An appraisal of published usability evaluations of electronic health records via systematic review

Ellsworth, Marc A.; Dziadzko, Mikhail; O’Horo, John C.; Farrell, Ann; Zhang, Jiajie; Herasevich, Vitaly

doi:10.1093/jamia/ocw046

Cited by 97 publications

(89 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This has been shown for intervention studies, where beliefs and expectations have been found to bias the results towards a higher probability of a type I error (i.e., false-positive result) [62]. The lack of methodological detail on the reports of studies on usability has already been highlighted [23,63]. The exponential growth and the enormous potential of mobile apps to change the paradigm of health interventions, by increasing the access of individuals to health services at lower costs, requires a rigorous and methodologically sound assessment.…”

Section: Discussionmentioning

confidence: 97%

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

Almeida

Rocha

Silva

2020

IJERPH

View full text Add to dashboard Cite

Background: There has been increasing use of mobile mHealth applications, including pain assessment and pain self-management apps. The usability of mHealth applications has vital importance as it affects the quality of apps. Thus, usability assessment with methodological rigor is essential to minimize errors and undesirable consequences, as well as to increase user acceptance. Objective: this study aimed to synthesize and evaluate existing studies on the assessment of the usability of pain-related apps using a newly developed scale. Methods: an electronic search was conducted in several databases, combining relevant keywords. Then titles and abstracts were screened against inclusion and exclusion criteria. The eligible studies were retrieved and independently screened for inclusion by two authors. Disagreements were resolved by discussion until consensus was reached. Results: a total of 31 articles were eligible for inclusion. Quality assessment revealed that most manuscripts did not assess usability using valid instruments or triangulation of methods of usability assessment. Most manuscripts also failed to assess the three domains of usability (effectiveness, efficiency and satisfaction). Conclusions: future studies should consider existing guidelines on usability assessment design, development and assessment of pain-related apps.

show abstract

Section: Discussionmentioning

confidence: 97%

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

Almeida

Rocha

Silva

2020

IJERPH

View full text Add to dashboard Cite

show abstract

“…From the methodological point of view, we found that survey or distributed questionnaires among end-users are the most common method employed in usability evaluations of health-related information systems (33). Beyond the usefulness of surveys for gathering data in these kinds of studies, they do not allow evaluators to identify individual usability problems, so, in considering this limitation, we applied heuristic evaluation in our study by recruiting 10 external evaluators to detect individual usability problems that could be targeted for improvement of the implemented system.…”

Section: Discussionmentioning

confidence: 99%

Evaluating the Usability of a Nationwide Pharmacy Information System in Iran: Application of Nielson’s Heuristics

Yasemi

Rahimi

Khajouei

2018

J Clin Res Paramed Sci

View full text Add to dashboard Cite

show abstract

“…Careful systematic development of CDS is necessary to ensure that it works as intended once implemented in practice. Researchers have found a number of problems in EHR usability testing (Ratwani, Benda, Hettinger, & Fairbanks, 2015) and a paucity of high-quality studies of EHR usability with two thirds performed at prepost implementation without preclinical usability testing reported (Ellsworth et al, 2017). In addition, half of the largest U.S. EHR vendors are not meeting standards for usability testing, with two thirds conducting tests with fewer than the minimum 15 participants, as suggested by the National Institute of Standards and Technology, and one fifth conducting at least half of their tests using subjects with no clinical background (Ratwani et al, 2015).…”

mentioning

confidence: 99%

Toward Meaningful Care Plan Clinical Decision Support

et al. 2017

View full text Add to dashboard Cite

Background Clinical decision support (CDS) tools—with easily understood and actionable information, at the point of care—are needed to help registered nurses (RNs) make evidence-based decisions. Not clear are the optimal formats of CDS tools. Thorough, preclinical testing is desirable to avoid costly errors associated with premature implementation in electronic health records. Objective To determine feasibility of the protocol for designed to compare multiple CDS formats, and evaluate effects of numeracy and graph literacy on RN adoption of best practices and care-planning time in a simulated environment. Methods In this pilot study, 60 RNs were randomly assigned to one of four CDS conditions (control; text; text+graph; text+table) and asked to adjust the plan of care for two patient scenarios over three shifts. A total of 14 best practices were identified for the two patients and sent as suggestions with evidence to the three CDS groups. Best practice adoption rates, care-planning time, and their relationship to the RN’s numeracy and graph literacy scores were assessed. Results CDS groups had a higher adoption rate of best practices (p < .001) across all shifts and decreased care-planning time in shifts two (p = .01) and three (p = .02) compared to the control group. Higher numeracy and graph literacy were associated with shorter care-planning times under text+table (p = .05) and text+graph conditions (p = .01). No significant differences were found between the three CDS groups on adoption rate and care-planning time. Discussion This pilot study demonstrates the feasibility of our protocol. Findings show preliminary evidence that CDS improves the efficiency and effectiveness of care-planning decisions, and that the optimal format may depend on individual RN characteristics. We recommend a study with sufficient power to compare different CDS formats, and assess the impact of potential covariates on adoption rates and care-planning time.

show abstract

An appraisal of published usability evaluations of electronic health records via systematic review

Cited by 97 publications

References 89 publications

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review

Evaluating the Usability of a Nationwide Pharmacy Information System in Iran: Application of Nielson’s Heuristics

Toward Meaningful Care Plan Clinical Decision Support

Contact Info

Product

Resources

About