Bonding With Bot: User Feedback on a Chatbot for Social Isolation

Abstract

Social isolation has affected people globally during the COVID-19 pandemic and had a major impact on older adult's well-being. Chatbot interventions may be a way to provide support to address loneliness and social isolation in older adults. The aims of the current study were to (1) understand the distribution of a chatbot's net promoter scores, (2) conduct a thematic analysis on qualitative elaborations to the net promoter scores, (3) understand the distribution of net promoter scores per theme, and (4) conduct a single word analysis to understand the frequency of words present in the qualitative feedback. A total of 7,099 adults and older adults consented to participate in a chatbot intervention on reducing social isolation and loneliness. The average net promoter score (NPS) was 8.67 out of 10. Qualitative feedback was provided by 766 (10.79%) participants which amounted to 898 total responses. Most themes were rated as positive (517), followed by neutral (311) and a minor portion as negative (70). The following five themes were found across the qualitative responses: positive outcome (277, 30.8%), user did not address question (262, 29.2%), bonding with the chatbot (240, 26.7%), negative technical aspects (70, 7.8%), and ambiguous outcome (49, 5.5%). Themes with a positive valence were found to be associated with a higher NPS. The word “help” and it's variations were found to be the most frequently used words, which is consistent with the thematic analysis. These results show that a chatbot for social isolation and loneliness was perceived positively by most participants. More specifically, users were likely to personify the chatbot (e.g., “Cause I feel like I have a new friend!”) and perceive positive personality features such as being non-judgmental, caring, and open to listen. A minor portion of the users reported dissatisfaction with chatting with a machine. Implications will be discussed.

Evaluating the Therapeutic Alliance With a Free-Text CBT Conversational Agent (Wysa): A Mixed-Methods Study

“…Notably, in the present study, many participants also made comments that personified the bot. These findings are aligned with results indicating that users of a chatbot intervention for social isolation and loneliness personified the chatbot and assigned human traits to it, such as being helpful, caring, open to listen, and non-judgmental ( 34 ). It is possible that user's personification of Wysa also indicates greater engagement.…”

“…In another study, researchers found that within 5 days of initial mental health application (app) use, users reported a therapeutic alliance with a conversational agent, measured through the Working Alliance Inventory-Short Revised (WAI-SR), which was comparable to those in recent studies from the literature on traditional, outpatient, individual and group CBT ( 33 ). Recently, Dosovitsky and Bunge ( 34 ) examined a chatbot intervention for social isolation and loneliness and found that most chatbot users were satisfied and would recommend the intervention to a friend. Their results showed a pattern of personifying the chatbot and assigning human traits to the chatbot (i.e., being helpful, caring, open to listen, and non-judgmental).…”

Mental Health Chatbot for Young Adults With Depressive Symptoms During the COVID-19 Pandemic: Single-Blind, Three-Arm Randomized Controlled Trial

“…This shows that XiaoE has preliminarily reached the standard of capacity in relationship establishment, but some aspects, such as the user interface and the operating system, still need to be further simplified for users. Participants reported having received the best experience with XiaoE in the 4 themes of “relationship,” “emotion,” “personalization,” and “practicality.” The theme of “relationship” reflected the establishment, development, and function of the relationship between XiaoE and participants, as Dosovitsky et al [ 52 ] found that individuals can form a positive bond with an AI chatbot owing to its personality traits, such as being caring, open to listening, and nonjudgmental. The theme of “emotion” reflected that communication with XiaoE was helpful for emotional expression and catharsis, and made users feel accompanied and understood.…”

“…Working alliance (also known as “therapeutic alliance”) represents the cooperative and emotionally connected relationship between the client and the therapist, and is considered a common factor in psychotherapy outcomes [ 49 ] and a metric to assess the computer-patient relationship as well [ 50 , 51 ]. Three recent studies [ 52 - 55 ] by Dosovitsky et al, Beatty et al, and Darcy et al had emphasized the viability and significance of the relationship and working alliance in digital treatment, and several randomized controlled trials of mental health chatbots had employed the Working Alliance Inventory (WAI) as a measurement method of working alliance, with all of these demonstrating good measure effects [ 37 , 56 , 57 ]. Important issues to be addressed for chatbots in the future could be extracted from the perceptions and opinions of patients [ 58 ], and thematic analysis with a topic model is a qualitative research method to accurately capture and concisely present key information in texts [ 59 ].…”

User experience with a parenting chatbot micro intervention

“…The research on chatbots for mental health problems in adults has shown that chatbots can produce some unique effects compared to other digital interventions. Users are able to engage and bond with chatbots ( 14 – 16 ) and tend to humanize and perceive the chatbot as their friend ( 17 ). Users have found mental health chatbots to be helpful, informative, easy to use ( 13 ), and have reported that chatbots are more accepting and not as judgemental as humans ( 16 ).…”