The Electronic Patient Reported Outcome Tool: Testing Usability and Feasibility of a Mobile App and Portal to Support Care for Patients With Complex Chronic Disease and Disability in Primary Care Settings

Abstract

BackgroundPeople experiencing complex chronic disease and disability (CCDD) face some of the greatest challenges of any patient population. Primary care providers find it difficult to manage multiple discordant conditions and symptoms and often complex social challenges experienced by these patients. The electronic Patient Reported Outcome (ePRO) tool is designed to overcome some of these challenges by supporting goal-oriented primary care delivery. Using the tool, patients and providers collaboratively develop health care goals on a portal linked to a mobile device to help patients and providers track progress between visits.ObjectivesThis study tested the usability and feasibility of adopting the ePRO tool into a single interdisciplinary primary health care practice in Toronto, Canada. The Fit between Individuals, Fask, and Technology (FITT) framework was used to guide our assessment and explore whether the ePRO tool is: (1) feasible for adoption in interdisciplinary primary health care practices and (2) usable from both the patient and provider perspectives. This usability pilot is part of a broader user-centered design development strategy.MethodsA 4-week pilot study was conducted in which patients and providers used the ePRO tool to develop health-related goals, which patients then monitored using a mobile device. Patients and providers collaboratively set goals using the system during an initial visit and had at least 1 follow-up visit at the end of the pilot to discuss progress. Focus groups and interviews were conducted with patients and providers to capture usability and feasibility measures. Data from the ePRO system were extracted to provide information regarding tool usage.ResultsSix providers and 11 patients participated in the study; 3 patients dropped out mainly owing to health issues. The remaining 8 patients completed 210 monitoring protocols, equal to over 1300 questions, with patients often answering questions daily. Providers and patients accessed the portal on an average of 10 and 1.5 times, respectively. Users found the system easy to use, some patients reporting that the tool helped in their ability to self-manage, catalyzed a sense of responsibility over their care, and improved patient-centered care delivery. Some providers found that the tool helped focus conversations on goal setting. However, the tool did not fit well with provider workflows, monitoring questions were not adequately tailored to individual patient needs, and daily reporting became tedious and time-consuming for patients.ConclusionsAlthough our study suggests relatively low usability and feasibility of the ePRO tool, we are encouraged by the early impact on patient outcomes and generally positive responses from both user groups regarding the potential of the tool to improve care for patients with CCDD. As is consistent with our user-centered design development approach, we have modified the tool based on user feedback, and are now testing the redeveloped tool through an exploratory trial.

Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems

“…Participants' scores from each test cycle may be compared with previous scores to confirm any improvements in satisfaction with the system. However, not all developers perceive usability scales as pertinent and some studies have suggested that a qualitative approach might be more useful especially in studies involving older participants [17,45]. Once again it is vital that the goals of the developers and stakeholders are considered when making decisions about using usability scales.…”

“…Recent research suggest that the use of ePRO systems could facilitate the remote monitoring of patients [13]; enhance efficiency by reducing the need for hospital appointments [14]; and improve patient outcomes such as quality of life and survival rates [15]. The number of health care providers developing ePRO systems has increased in recent years [16,17] and is set to rise considerably in future as more evidence to support their use become available.…”

Using Exploratory Trials to Identify Relevant Contexts and Mechanisms in Complex Electronic Health Interventions: Evaluating the Electronic Patient-Reported Outcome Tool (Preprint)

“…Consistent with the developmental evaluation approach, informed by our overarching user-centred design methods, data from this study is also used to inform design changes to the technology (see prior publications on development and usability testing [21,22]). Table 3 summarizes outcome, process and outcome variables of interest to our study, and the tools/methods used to capture data.…”

A mHealth-Intervention to Support the Self-Management of Multimorbid Patients in General Practices in Germany (Telepracman) – Study Protocol of a Randomized Pilot Study

“…In the research literature, we identi ed only one study on a mHealth-intervention that explicitly addressed multimorbid patients in the primary care setting [13][14][15][16][17][18]. This study showed that multimorbid patients accept and use mHealth-interventions.…”

“…Furthermore, the intervention was able to support them in selfmanagement [17,19]. Nevertheless, there arose challenges of successfully integrating the intervention into the patients' everyday life as well as into the work ows of the GP-practices [17,18]. Signi cant changes in patient outcomes referring to quality of life and activation could not be shown in a fourmonths trail [17].…”