Agile, Easily Applicable, and Useful eHealth Usability Evaluations: Systematic Review and Expert-Validation

Abstract

Background Electronic health (eHealth) usability evaluations of rapidly developed eHealth systems are difficult to accomplish because traditional usability evaluation methods require substantial time in preparation and implementation. This illustrates the growing need for fast, flexible, and cost-effective methods to evaluate the usability of eHealth systems. To address this demand, the present study systematically identified and expert-validated rapidly deployable eHealth usability evaluation methods. Objective Identification and prioritization of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations. Methods The study design comprised a systematic iterative approach in which expert knowledge was contrasted with findings from literature. Forty-three eHealth usability evaluation methods were systematically identified and assessed regarding their ease of applicability and usefulness through semi-structured expert interviews with 10 European usability experts and systematic literature research. The most appropriate eHealth usability evaluation methods were selected stepwise based on the experts' judgements of their ease of applicability and usefulness. Results Of these 43 eHealth usability evaluation methods identified as suitable for agile, easily applicable, and useful eHealth usability evaluations, 10 were recommended by the experts based on their usefulness for rapid eHealth usability evaluations. The three most frequently recommended eHealth usability evaluation methods were Remote User Testing, Expert Review, and Rapid Iterative Test and Evaluation Method. Eleven usability evaluation methods, such as Retrospective Testing, were not recommended for use in rapid eHealth usability evaluations. Conclusion We conducted a systematic review and expert-validation to identify rapidly deployable eHealth usability evaluation methods. The comprehensive and evidence-based prioritization of eHealth usability evaluation methods supports faster usability evaluations, and so contributes to the ease-of-use of emerging eHealth systems.

The “Breast-4Y” web app for breast cancer prevention at young age: Development, evaluation, and validation

“…Usability refers to “ the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use ( International Organization for Standardization, 2018 ).” Assessing usability of digital resources is essential because of the relationship between high usability and easier tool usage ( Maramba et al, 2019 ; Sinabell and Ammenwerth, 2022 ), adherence to web-app based interventions ( Jakob et al, 2022 ), effectiveness and efficiency to achieve the expected goals ( Klaassen et al, 2016 ) and increase patient safety ( Kushniruk and Borycki, 2015 ; Kushniruk and Borycki, 2017 ; Marcilly et al, 2019 ). Results of our study showed that no differences were observed in the quality and usability scores due to educational level nor age.…”

Agile Software Development in Healthcare: A Synthetic Scoping Review

“…Agile user-centred design of smart eHealth application: the importance of user-centred design in developing medical software and eHealth solutions was recognised about 15 years ago [83]. In general, the research claims that user-centred design should also be used in ASD in healthcare.…”

Improving the User Experience with Discount Site-Specific User Testing

“…The usability methods described in this report align with a recent systematic review of health-related usability evaluations. 24 Among the methods recommended by experts and employed in our program include remote user testing (combined with think aloud or interview), heuristic evaluation, and A/B testing. In an example not highlighted in this report, A/B testing gave a clear indication of a preferred option for an acute opioid clinical decision support alert in the first test event.…”

“…A recent review cites the SUS as an example of a questionnaire "not recommended" as an EHR usability evaluation method. 24 Whether and in what contexts the SUS adds value to site-specific user testing remains to be determined. Finally, testing outside of formal "test events" may help an organization scale up a testing program by simplifying recruitment.…”