Using Natural Language Processing and Sentiment Analysis to Augment Traditional User-Centered Design: Development and Usability Study

Abstract

Background Sarcopenia, defined as the age-associated loss of muscle mass and strength, can be effectively mitigated through resistance-based physical activity. With compliance at approximately 40% for home-based exercise prescriptions, implementing a remote sensing system would help patients and clinicians to better understand treatment progress and increase compliance. The inclusion of end users in the development of mobile apps for remote-sensing systems can ensure that they are both user friendly and facilitate compliance. With advancements in natural language processing (NLP), there is potential for these methods to be used with data collected through the user-centered design process. Objective This study aims to develop a mobile app for a novel device through a user-centered design process with both older adults and clinicians while exploring whether data collected through this process can be used in NLP and sentiment analysis Methods Through a user-centered design process, we conducted semistructured interviews during the development of a geriatric-friendly Bluetooth-connected resistance exercise band app. We interviewed patients and clinicians at weeks 0, 5, and 10 of the app development. Each semistructured interview consisted of heuristic evaluations, cognitive walkthroughs, and observations. We used the Bing sentiment library for a sentiment analysis of interview transcripts and then applied NLP-based latent Dirichlet allocation (LDA) topic modeling to identify differences and similarities in patient and clinician participant interviews. Sentiment was defined as the sum of positive and negative words (each word with a +1 or −1 value). To assess utility, we used quantitative assessment questionnaires—System Usability Scale (SUS) and Usefulness, Satisfaction, and Ease of use (USE). Finally, we used multivariate linear models—adjusting for age, sex, subject group (clinician vs patient), and development—to explore the association between sentiment analysis and SUS and USE outcomes. Results The mean age of the 22 participants was 68 (SD 14) years, and 17 (77%) were female. The overall mean SUS and USE scores were 66.4 (SD 13.6) and 41.3 (SD 15.2), respectively. Both patients and clinicians provided valuable insights into the needs of older adults when designing and building an app. The mean positive-negative sentiment per sentence was 0.19 (SD 0.21) and 0.47 (SD 0.21) for patient and clinician interviews, respectively. We found a positive association with positive sentiment in an interview and SUS score (ß=1.38; 95% CI 0.37 to 2.39; P=.01). There was no significant association between sentiment and the USE score. The LDA analysis found no overlap between patients and clinicians in the 8 identified topics. Conclusions Involving patients and clinicians allowed us to design and build an app that is user friendly for older adults while supporting compliance. This is the first analysis using NLP and usability questionnaires in the quantification of user-centered design of technology for older adults.

Human-Centered Design of Mobile Health Apps for Older Adults: Systematic Review and Narrative Synthesis

“…One study aimed at patients with psychiatric disorders [ 8 ]. The other 2 studies dealt with fall risk assessment and detection [ 32 ] and sarcopenia prevention [ 34 ], respectively. In addition, of the 3 included qualitative case studies, 2 reflected on the same project as the mixed methods study dealing with falls in Ireland [ 33 , 36 ].…”

“…All 8 studies are summarized and described in 4 HCD steps. Each step has 5 paragraphs representing a total of 5 mHealth projects: the first for patients with heart failure [ 31 , 35 ], the second for patients with psychiatric disorders [ 8 ], the third about falls in the elderly [ 32 , 33 , 36 ], the fourth for sarcopenia prevention [ 34 ], and the fifth about CIEDs [ 30 ].…”

“…Petersen et al [ 34 ] used HCD to develop an mHealth app featuring exercise videos to work with a Bluetooth-connected resistance band that together would help health care providers monitor older adults’ exercise progress for sarcopenia prevention. A convenience sampling method was used.…”

“…Petersen et al [ 34 ] concluded pain points from their previous research regarding the exercise videos as follows: specific movements were hard to identify from low-contrast backgrounds, and instruction sounds were not heard clearly. Participants also stated that big and clear repetition numbers would help them better keep track, feedback and instructions would help them finish the exercises at home, and tablets were preferred as they have large screens.…”

“…Petersen et al [ 34 ] updated the exercise videos from user pain points and created mobile prototypes as black and white wireframes showing simple outlines of the designed user interfaces. The researchers then presented exercise videos and the wireframes with different design approaches to 3 clinicians and 4 older patients from an academic health center in the United States for additional inputs.…”