A Mobile Patient-Reported Outcome Measure App With Talking Touchscreen: Usability Assessment

Abstract

BackgroundIn the past years, a mobile health (mHealth) app called the Dutch Talking Touch Screen Questionnaire (DTTSQ) was developed in The Netherlands. The aim of development was to enable Dutch physical therapy patients to autonomously complete a health-related questionnaire regardless of their level of literacy and digital skills.ObjectiveThe aim of this study was to evaluate the usability (defined as the effectiveness, efficiency, and satisfaction) of the prototype of the DTTSQ for Dutch physical therapy patients with diverse levels of experience in using mobile technology.MethodsThe qualitative Three-Step Test-Interview method, including both think-aloud and retrospective probing techniques, was used to gain insight into the usability of the DTTSQ. A total of 24 physical therapy patients were included. The interview data were analyzed using a thematic content analysis approach aimed at analyzing the accuracy and completeness with which participants completed the questionnaire (effectiveness), the time it took the participants to complete the questionnaire (efficiency), and the extent to which the participants were satisfied with the ease of use of the questionnaire (satisfaction). The problems encountered by the participants in this study were given a severity rating that was used to provide a rough estimate of the need for additional usability efforts.ResultsAll participants within this study were very satisfied with the ease of use of the DTTSQ. Overall, 9 participants stated that the usability of the app exceeded their expectations. The group of 4 average-/high-experienced participants encountered only 1 problem in total, whereas the 11 little-experienced participants encountered an average of 2 problems per person and the 9 inexperienced participants an average of 3 problems per person. A total of 13 different kind of problems were found during this study. Of these problems, 4 need to be addressed before the DTTSQ will be released because they have the potential to negatively influence future usage of the tool. The other 9 problems were less likely to influence future usage of the tool substantially.ConclusionsThe usability of the DTTSQ needs to be improved before it can be released. No problems were found with satisfaction or efficiency during the usability test. The effectiveness needs to be improved by (1) making it easier to navigate through screens without the possibility of accidentally skipping one, (2) enabling the possibility to insert an answer by tapping on the text underneath a photograph instead of just touching the photograph itself, and (3) making it easier to correct wrong answers. This study shows the importance of including less skilled participants in a usability study when striving for inclusive design and the importance of measuring not just satisfaction but also efficiency and effectiveness during such studies.

User-Centered Development and Testing of the Online Patient-Reported Outcomes, Burdens, and Experiences (PROBE) Survey and the myPROBE App and Integration With the Canadian Bleeding Disorder Registry: Mixed Methods Study

“…In terms of mobile apps for collecting PRO data in patients with chronic conditions other than hemophilia, previous studies reported conflicting results. Welbie et al tested the usability of an app for PROs in physical therapy patients and found that users where overall satisfied with the usability of the app, but the app required some changes in navigation through questions and how to insert and edit answers [ 19 ]. The authors acknowledged that they would again test the usability of the app after implementing such changes.…”

“…Different solutions have been proposed for the digital collection of PRO measures, including modifying the architecture of the electronic health record to integrate data collection from different mobile apps [ 17 ]. However, once implemented, the usability of these systems needs to be tested, as changes might be needed if the results are unsatisfactory [ 18 , 19 ].…”

Evaluating the usability of a cancer registry system using Cognitive Walkthrough, and assessing user agreement with its problems

“…To determine the severity of usability problems of information systems, among several severity rating method [ 34 – 36 ], we used a common severity rating method proposed by Nielsen that prioritizes the problems in terms of their effect on user interaction [ 31 , 36 – 39 ]. In this method, the severity of problems is determined based on three criteria of frequency, impact, and persistence [ 40 ].…”

The Review of Usability Evaluation Methods on Tele health or Telemedicine Systems

“…There are several methods of usability evaluation that include Semi-structured interviews, Eye-tracking, Contextual interview, Cognitive walkthrough, Think-aloud, Focus group discussion, Heuristic Evaluation and expert review, Scenario, Questionnaires. There are many forms of standardized questionnaires such as mobile health APP usability questionnaire (MAUQ), System Usability Scale (SUS), Telehealth Usability Questionnaire, IBM ease of use questionnaires, Software Usability Measurement Inventory (SUMI), Technology Acceptance Model-2 (TAM-2), Health Information Technology Usability valuation Scale (Health-ITUES), NASA Task Load Index (NASA TLX), the Post-Study System Usability Questionnaire (PSSUQ), and Questionnaire For User Interaction Satisfaction(QUIS) [17,18]. This study aims to survey usability evaluation methods on tele health or telemedicine systems that run on the Web or Android platform and determine the variety of usability attributes in them.…”