Going Remote—Demonstration and Evaluation of Remote Technology Delivery and Usability Assessment With Older Adults: Survey Study

Abstract

Background The COVID-19 pandemic necessitated “going remote” with the delivery, support, and assessment of a study intervention targeting older adults enrolled in a clinical trial. While remotely delivering and assessing technology is not new, there are few methods available in the literature that are proven to be effective with diverse populations, and none for older adults specifically. Older adults comprise a diverse population, including in terms of their experience with and access to technology, making this a challenging endeavor. Objective Our objective was to remotely deliver and conduct usability testing for a mobile health (mHealth) technology intervention for older adult participants enrolled in a clinical trial of the technology. This paper describes the methodology used, its successes, and its limitations. Methods We developed a conceptual model for remote operations, called the Framework for Agile and Remote Operations (FAR Ops), that combined the general requirements for spaceflight operations with Agile project management processes to quickly respond to this challenge. Using this framework, we iteratively created care packages that differed in their contents based on participant needs and were sent to study participants to deliver the study intervention—a medication management app—and assess its usability. Usability data were collected using the System Usability Scale (SUS) and a novel usability questionnaire developed to collect more in-depth data. Results In the first 6 months of the project, we successfully delivered 21 care packages. We successfully designed and deployed a minimum viable product in less than 6 weeks, generally maintained a 2-week sprint cycle, and achieved a 40% to 50% return rate for both usability assessment instruments. We hypothesize that lack of engagement due to the pandemic and our use of asynchronous communication channels contributed to the return rate of usability assessments being lower than desired. We also provide general recommendations for performing remote usability testing with diverse populations based on the results of our work, including implementing screen sharing capabilities when possible, and determining participant preference for phone or email communications. Conclusions The FAR Ops model allowed our team to adopt remote operations for our mHealth trial in response to interruptions from the COVID-19 pandemic. This approach can be useful for other research or practice-based projects under similar circumstances or to improve efficiency, cost, effectiveness, and participant diversity in general. In addition to offering a replicable approach, this paper tells the often-untold story of practical challenges faced by mHealth projects and practical strategies used to address them. Trial Registration ClinicalTrials.gov NCT04121858; https://clinicaltrials.gov/ct2/show/NCT04121858

Feasibility and lessons learned on remote trial implementation from TestBoston, a fully remote, longitudinal, large-scale COVID-19 surveillance study

“…While remote trials may alleviate structural barriers by allowing flexibility based on personal availability, TestBoston participants highlighted work and family obligations, time, and scheduling constraints that conflicted with sample return, as significant barriers. Other potential gaps that have been described include technology access, literacy and privacy concerns that may be unaddressed, and at worst, exacerbated by remote models (14,(39)(40)(41)(42). We attempted to mitigate these factors by providing translation services, intense participant support, and participantinformed adaptations.…”

Reaching People Where They Are: Remote Macroergonomics Research

“…Sprints: Conducting rapid-cycle tests of prototype solutions Sensors: Collecting timely, actionable, nonjudgmental feedback to inform learning and redesign Psychological Safety: Establishing rituals and artifacts to maintain a culture in which people can be authentic without fear Scientific Method: Accepting evidence as the basis for decision making I will present examples of our Agile approach in action, using illustrative cases from our organization's initial work to take our study operations "remote" (Hill et al, 2021) and continuous improvement work to increase the efficiency and effectiveness of our RCTs. I will discuss challenges, implementation strategies, theoretical underpinnings, and recommendations for broader adoption of Agile into research organizations.…”

“…Researchers were not exempt from having to learn to "go remote". Many studies involving human subjects transitioned to remote operations at the start of the pandemic and many still retain remote operations today-for examples see (Hill et al, 2021;Inverso et al, 2021;Lai et al, 2020). Increasing our understanding of how human subjects research can be performed outside of traditional face-to-face interactions can be beneficial.…”

The Patient in Patient Safety: Contemporary Issues, From COVID-19 to Citizen Science

“…Hill and Dr. Brown discussed their work to overcome the challenges to human subjects research created by the suspension of face-to-face activities during the COVID-19 pandemic. First, the development of "care packages" to enable remote study intervention delivery and usability assessment with older adults was described-more details available in Hill et al (2021). Following this, Dr. Brown described her work to increase the engagement of older adult study participants.…”