Conceptualizing Usability for the eHealth Context: Content Analysis of Usability Problems of eHealth Applications (Preprint)

Abstract

BACKGROUND Usability tests can be either formative (where the aim is to detect usability problems) or summative (where the aim is to benchmark usability). There are ample formative methods that can take into account user characteristics and contexts (i.e., cognitive walkthrough, co-discovery, verbal protocols). This is especially valuable for eHealth applications, as health conditions can influence user-technology interaction. However, most summative usability tests do not consider eHealth-specific factors that could potentially affect the usability of a system. One of the reasons for this, is that there are currently no fine-grained frameworks or models of usability factors that are unique for the eHealth domain. OBJECTIVE The goal of this study is to develop an ontology of usability problems, specifically for eHealth applications with patients as their primary end-user group. METHODS We analysed eight datasets, containing the results of eight formative usability tests of eHealth applications. These datasets contained 400 usability problems that were usable for analysis. Both, inductive and deductive coding was used to create the ontology. Six datasets were used to create the ontology; two datasets were used for the validation of the framework by assessing the inter-coder agreement. RESULTS Eight main categories of usability factors were identified: (1) System Basic Performance, (2) Task-Technology Fit, (3) User Accommodativeness, (4) Interface Design, (5) Navigation & Structure, (6) Information & Terminology, (7) Guidance & Support, (8) Satisfaction. These eight categories contain a total of 21 factors: 14 general usability factors and seven context-specific factors. Cohen’s kappa was calculated for two datasets, on both category and factor level. The Kappa’s were all between k = 0.62 and k = 0.67, which is acceptable. Descriptive analysis revealed that approximately 70% of the usability problems can be considered as general usability factors and 30% as eHealth-specific usability factors. CONCLUSIONS Our ontology provides a detailed overview of usability factors for eHealth applications. Current usability benchmarking instruments include only a subset of the factors that emerged from our study and are therefore not fully suited for summative evaluations of eHealth applications. Our findings can support the development of new usability benchmarking tools for the eHealth domain.

Perspectives on Mental Health Digital Platforms: A Usability Study with Ergonomics Design Students

“…Mental health apps, designed to offer a plethora of resources ranging from psychoeducation to guided therapeutic exercises, have the potential to transcend these barriers. However, the effectiveness of such apps hinges critically on an often overlooked aspect of usability yet best usability practices are often lacking for many mental health apps 46,47,63 . User reviews and feedback can help identify issues with helpfulness, engagement and interface design that impact usability 49 64 .…”

Perspectives on Mental Health Digital Platforms: A Usability Study with Ergonomics Design Students

“…Mental health apps, designed to offer a plethora of resources ranging from psychoeducation to guided therapeutic exercises, have the potential to transcend these barriers. However, the effectiveness of such apps hinges critically on an often overlooked aspect of usability yet best usability practices are often lacking for many mental health apps 47,48,64 . User reviews and feedback can help identify issues with helpfulness, engagement and interface design that impact usability 50 65 .…”