Understanding User Experience: Exploring Participants’ Messages With a Web-Based Behavioral Health Intervention for Adolescents With Chronic Pain

Abstract

Background Delivery of behavioral health interventions on the internet offers many benefits, including accessibility, cost-effectiveness, convenience, and anonymity. In recent years, an increased number of internet interventions have been developed, targeting a range of conditions and behaviors, including depression, pain, anxiety, sleep disturbance, and eating disorders. Human support (coaching) is a common component of internet interventions that is intended to boost engagement; however, little is known about how participants interact with coaches and how this may relate to their experience with the intervention. By examining the data that participants produce during an intervention, we can characterize their interaction patterns and refine treatments to address different needs. Objective In this study, we employed text mining and visual analytics techniques to analyze messages exchanged between coaches and participants in an internet-delivered pain management intervention for adolescents with chronic pain and their parents. Methods We explored the main themes in coaches’ and participants’ messages using an automated textual analysis method, topic modeling. We then clustered participants’ messages to identify subgroups of participants with similar engagement patterns. Results First, we performed topic modeling on coaches’ messages. The themes in coaches’ messages fell into 3 categories: Treatment Content, Administrative and Technical, and Rapport Building. Next, we employed topic modeling to identify topics from participants’ message histories. Similar to the coaches’ topics, these were subsumed under 3 high-level categories: Health Management and Treatment Content, Questions and Concerns, and Activities and Interests. Finally, the cluster analysis identified 4 clusters, each with a distinguishing characteristic: Assignment-Focused, Short Message Histories, Pain-Focused, and Activity-Focused. The name of each cluster exemplifies the main engagement patterns of that cluster. Conclusions In this secondary data analysis, we demonstrated how automated text analysis techniques could be used to identify messages of interest, such as questions and concerns from users. In addition, we demonstrated how cluster analysis could be used to identify subgroups of individuals who share communication and engagement patterns, and in turn facilitate personalization of interventions for different subgroups of patients. This work makes 2 key methodological contributions. First, this study is innovative in its use of topic modeling to provide a rich characterization of the textual content produced by coaches and participants in an internet-delivered behavioral health intervention. Second, to our knowledge, this is the first example of the use of a visual analysis method to cluster participants and identify similar patterns of behavior based on intervention message content.

Suitability of Text-Based Communications for the Delivery of Psychological Therapeutic Services to Rural and Remote Communities: Scoping Review

“…A study of an internet-delivered pain management intervention for adolescents with chronic pain [90] used text mining and analytic techniques to messages between coaches and patients and successfully identified messages that raised concerns. Cluster analysis identified subgroups of individuals with common communication and engagement patterns that could be used to tailor interventions.…”

Machine and cognitive intelligence for human health: systematic review

“…Schäfer et al [ 60 ] identified “Web-based discussion topics associated with Gastrointestinal discomfort and its perceived factors in Web-based messages posted by users of French social media (p. 1)” using topic modeling to provide real-world evidence for caregivers. Chen et al [ 61 ] employed topic modeling and visual analytics techniques to characterize textual content generated during Internet behavioral health interventions. In [ 62 ], an RNN-based semi-supervised learning algorithm exploited rich unlabeled Web corpus.…”

“…In terms of algorithm and method innovations, future efforts include: (1) using knowledge graphs to analyze medical information strongly relevant to expert knowledge to boost prediction performance [ 86 ], (2) adopting ontology-based methods to perform complex plan-oriented counseling and communication tasks [ 84 ], (3) utilizing CNNs and LSTM-CNN with diverse embedding and optimization technologies for epidemic outbreak analysis [ 44 ], (4) applying semi-automatic approaches to promote personalized healthcare information provided to facilitate users’ daily activities of living [ 76 ], (5) integrating additional security technologies like hashing to avoid malicious attackers [ 57 ], (6) using a combination of algorithms such as genetic algorithms and SVMs to facilitate accurate feature selection [ 55 ], and (7) adding more privacy-preserving statistics and machine learning algorithms to extensively promote flexibility in secure multicenters [ 57 ]. Additionally, there are scholars indicating the need to: (1) propose visual approaches to explore “the dyadic interaction between coaches and participants to better understand how to provide support and guidance to participants (p. 14) [ 61 ]”, and (2) analyze, mine, and extract Web page content by adopting machine learning algorithms and through quality information visualization within search engines [ 87 ].…”

“…Third, Sahu et al [ 90 ] mentioned collecting “demographic information from a subset of existing ecobee users to understand the association between age, sex, and other relevant demographic indicators (p. 8)”, and Arsevska et al [ 79 ] indicated integrating geographical and language factors. Other directions include: (1) integrating additional physiological signal monitoring modules [ 91 ], (2) combining the temporality of messages in clustering [ 61 ], (3) adopting linked open data as complementary answer sources [ 30 ], (4) exploiting health data produced via passive smartphone sensing technologies and linking them with Web-based applications [ 92 ], (5) integrating additional types of mappings or services with the basis of clinical guidelines to allow linking electronic health records with guideline-oriented decision support applications [ 93 ], (6) integrating multiple context information based on deep learning [ 94 ], (7) allowing seamless integration of data from varied sources or repositories [ 54 ], and (8) collecting propagation-related information and time series information to enhance model performance [ 86 ].…”

Online multidisciplinary interventions for paediatric chronic pain: A content analysis

“…Overall findings indicated a high level of efficacy. Several secondary analyses of the data from the main Web‐MAP2 trial have been conducted (Chen et al., 2019; Law et al., 2018; Murray et al., 2019).…”