Untold Stories in User-Centered Design of Mobile Health: Practical Challenges and Strategies Learned From the Design and Evaluation of an App for Older Adults With Heart Failure

Cornet, Victor P.; Toscos, Tammy; Bolchini, Davide; Ghahari, Romisa Rohani; Ahmed, Ryan; Daley, Carly; Mirro, Michael J.; Holden, Richard J.

doi:10.2196/17703

Cited by 73 publications

(80 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Design for older adults is typically a multidimensional process involving significant time and cost in thinking, problem-solving research, iterative testing, and redesigning to meet the needs, capabilities, and limitations of users [ 37 ]. However, there are still great opportunities to be discussed and learned from the untold stories of implementing a user-centered design to create more efficient, effective, and sustainable eHealth solutions [ 38 ]. The COVID-19 pandemic highlights the significant value of digital technologies for reaching older adults, especially if frail and with multiple chronic diseases.…”

Section: Discussionmentioning

confidence: 99%

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Stara¹,

Vera²,

Bolliger³

et al. 2020

JMIR mHealth and uHealth

View full text Add to dashboard Cite

Background Information and communication technologies are tools that are able to support cognitive functions, monitor health and movements, provide reminders to maintain residual memory abilities, and promote social support, especially among patients with dementia. Among these technologies, embodied conversational agents (ECAs) are seen as screen-based entities designed to stimulate human face-to-face conversation skills, allowing for natural human-machine interaction. Unfortunately, the evidence that such agents deliver care benefits in supporting people affected by dementia and their caregivers has not yet been well studied. Therefore, research in this area is essential for the entire scientific community. Objective This study aims to evaluate the usability and acceptability of the virtual agent Anne by people living with dementia. The study is also designed to assess the ability of target users to use the system independently and receive valuable information from it. Methods We conducted a 4-week trial that involved 20 older adults living with dementia and 14 family caregivers in home environment settings in Italy. This study used a mixed methods approach, balancing quantitative and qualitative instruments to gather data from users. Telemetry data were also collected. Results Older users were particularly engaged in providing significant responses and participating in system improvements. Some of them clearly discussed how technical problems related to speech recognition had a negative impact on the intention to use, adaptiveness, usefulness, and trust. Moreover, the usability of the system achieved an encouraging score, and half of the sample recognized a role of the agent Anne. This study confirms that the quality of automatic speech recognition and synthesis is still a technical issue and has room for improvement, whereas the touch screen modality is almost stable and positively used by patients with dementia. Conclusions This study demonstrated the ability of target users to use the system independently in their home environment; overall, the involved participants shared good engagement with the system, approaching the virtual agents as a companion able to support memory and enjoyment needs. Therefore, this research provides data that sustain the use of ECAs as future eHealth systems that are able to address the basic and higher-level needs of people living with dementia. This specific field of research is novel and poorly discussed in the scientific community. This could be because of its novelty, yet there is an urgent need to strengthen data, research, and innovation to accelerate the implementation of ECAs as a future method to offer nonpharmacological support to community-dwelling people with dementia.

show abstract

Section: Discussionmentioning

confidence: 99%

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Stara¹,

Vera²,

Bolliger³

et al. 2020

JMIR mHealth and uHealth

View full text Add to dashboard Cite

show abstract

“…In addition to offering a replicable approach, this paper tells the often-untold story of practical challenges faced by mHealth projects and practical strategies used, as another article in this journal [ 45 ] compels us to do:…”

Section: Discussionmentioning

confidence: 99%

Going Remote—Demonstration and Evaluation of Remote Technology Delivery and Usability Assessment With Older Adults: Survey Study

Hill¹,

Harrington²,

Adeoye³

et al. 2021

JMIR Mhealth Uhealth

Self Cite

View full text Add to dashboard Cite

Background The COVID-19 pandemic necessitated “going remote” with the delivery, support, and assessment of a study intervention targeting older adults enrolled in a clinical trial. While remotely delivering and assessing technology is not new, there are few methods available in the literature that are proven to be effective with diverse populations, and none for older adults specifically. Older adults comprise a diverse population, including in terms of their experience with and access to technology, making this a challenging endeavor. Objective Our objective was to remotely deliver and conduct usability testing for a mobile health (mHealth) technology intervention for older adult participants enrolled in a clinical trial of the technology. This paper describes the methodology used, its successes, and its limitations. Methods We developed a conceptual model for remote operations, called the Framework for Agile and Remote Operations (FAR Ops), that combined the general requirements for spaceflight operations with Agile project management processes to quickly respond to this challenge. Using this framework, we iteratively created care packages that differed in their contents based on participant needs and were sent to study participants to deliver the study intervention—a medication management app—and assess its usability. Usability data were collected using the System Usability Scale (SUS) and a novel usability questionnaire developed to collect more in-depth data. Results In the first 6 months of the project, we successfully delivered 21 care packages. We successfully designed and deployed a minimum viable product in less than 6 weeks, generally maintained a 2-week sprint cycle, and achieved a 40% to 50% return rate for both usability assessment instruments. We hypothesize that lack of engagement due to the pandemic and our use of asynchronous communication channels contributed to the return rate of usability assessments being lower than desired. We also provide general recommendations for performing remote usability testing with diverse populations based on the results of our work, including implementing screen sharing capabilities when possible, and determining participant preference for phone or email communications. Conclusions The FAR Ops model allowed our team to adopt remote operations for our mHealth trial in response to interruptions from the COVID-19 pandemic. This approach can be useful for other research or practice-based projects under similar circumstances or to improve efficiency, cost, effectiveness, and participant diversity in general. In addition to offering a replicable approach, this paper tells the often-untold story of practical challenges faced by mHealth projects and practical strategies used to address them. Trial Registration ClinicalTrials.gov NCT04121858; https://clinicaltrials.gov/ct2/show/NCT04121858

show abstract

“…Design for older adults is typically a multi-dimensional process involving significant time and cost in thinking, problem-solving research, iterative testing and re-design to meet the needs, capabilities and limitations of users [37]. However, there are still great opportunities to be discussed and learned from the untold stories of implementing UCD to create more efficient, effective, and sustainable e-Health solutions [38].…”

Section: Discussionmentioning

confidence: 99%

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Stara¹,

Vera²,

Bolliger³

et al. 2020

Preprint

View full text Add to dashboard Cite

BACKGROUND Information and Communication Technologies are seen as tools able to support cognitive functions, monitor health and movements, provide reminders to maintain residual memory abilities and promote social support, especially among patients with dementia. Among these technologies, Embodied Conversational Agents (ECAs) are seen as screen-based entities, designed to stimulate human face-to-face conversation skills and thus allow for natural human-machine interaction. Unfortunately, the efficacy of ECA in supporting people affected by dementia and their caregivers is not yet well studied. Therefore, research in this area is essential for the entire scientific community. OBJECTIVE This study aims to evaluate the usability and acceptance of the ECA Anne by seniors affected with dementia. The study is also designed to assess the ability of target users to utilize the system independently and receive valuable information from it. METHODS A 4-week trial was conducted involving 20 older adults with dementia and 14 family caregivers in home environment settings in Italy. This study used a mixed method approach, balancing quantitative and qualitative instruments to gather data from users. Telemetry data was also collected. RESULTS It emerges that older users were particularly engaged in providing significant responses and participated in system improvements. Some of them clearly discussed how technical problems related to speech recognition negatively impacted on the intention to use, adaptiveness, usefulness and trust. Moreover, the usability of the system achieved an encouraging score and the half of the sample recognized a role in the ECA. This study confirms that the quality of automatic speech recognition and synthesis is still a technical issue and has room for improvement, whereas touchscreen modality is almost stable and positively used by patients with dementia. CONCLUSIONS This specific field of research is novel and poorly discussed in the scientific community. This could be due to its newness, yet there is an urgent need to strengthen data, research and innovation to accelerate the implementation of ECA as a future way to offer non-pharmacological support to community-dwelling persons with dementia.

show abstract

Untold Stories in User-Centered Design of Mobile Health: Practical Challenges and Strategies Learned From the Design and Evaluation of an App for Older Adults With Heart Failure

Cited by 73 publications

References 81 publications

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Going Remote—Demonstration and Evaluation of Remote Technology Delivery and Usability Assessment With Older Adults: Survey Study

Usability and Acceptance of the Embodied Conversational Agent Anne by People with Dementia and their Caregivers: an exploratory study in home environment settings (Preprint)

Contact Info

Product

Resources

About