Background Rapid proliferation of mental health interventions delivered through conversational agents (CAs) calls for high-quality evidence to support their implementation and adoption. Selecting appropriate outcomes, instruments for measuring outcomes, and assessment methods are crucial for ensuring that interventions are evaluated effectively and with a high level of quality. Objective We aimed to identify the types of outcomes, outcome measurement instruments, and assessment methods used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health. Methods We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and assessment methods in studies that evaluated the effectiveness of CA interventions for mental health. We performed a comprehensive search of electronic databases, including PubMed, Cochrane Central Register of Controlled Trials, Embase (Ovid), PsychINFO, and Web of Science, as well as Google Scholar and Google. We included experimental studies evaluating CA mental health interventions. The screening and data extraction were performed independently by 2 review authors in parallel. Descriptive and thematic analyses of the findings were performed. Results We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (21/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical outcomes (123/203, 60.6%), user experience outcomes (75/203, 36.9%), technical outcomes (2/203, 1.0%), and other outcomes (3/203, 1.5%). Most of the outcome measurement instruments were used in only 1 study (150/203, 73.9%) and were self-reported questionnaires (170/203, 83.7%), and most were delivered electronically via survey platforms (61/203, 30.0%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 52.7%), which were largely created or adapted for the study in which they were used (95/107, 88.8%). Conclusions The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health point to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by CAs and smartphones to streamline the evaluation and reduce participants’ input burden inherent to self-reporting.

BackgroundNon-communicable diseases (NCDs) and common mental disorders (CMDs) are the leading causes of death and disability worldwide. Lifestyle interventions via mobile apps and conversational agents present themselves as low-cost, scalable solutions to prevent these conditions. This paper describes the rationale for, and development of, “LvL UP 1.0″, a smartphone-based lifestyle intervention aimed at preventing NCDs and CMDs.Materials and MethodsA multidisciplinary team led the intervention design process of LvL UP 1.0, involving four phases: (i) preliminary research (stakeholder consultations, systematic market reviews), (ii) selecting intervention components and developing the conceptual model, (iii) whiteboarding and prototype design, and (iv) testing and refinement. The Multiphase Optimization Strategy and the UK Medical Research Council framework for developing and evaluating complex interventions were used to guide the intervention development.ResultsPreliminary research highlighted the importance of targeting holistic wellbeing (i.e., both physical and mental health). Accordingly, the first version of LvL UP features a scalable, smartphone-based, and conversational agent-delivered holistic lifestyle intervention built around three pillars: Move More (physical activity), Eat Well (nutrition and healthy eating), and Stress Less (emotional regulation and wellbeing). Intervention components include health literacy and psychoeducational coaching sessions, daily “Life Hacks” (healthy activity suggestions), breathing exercises, and journaling. In addition to the intervention components, formative research also stressed the need to introduce engagement-specific components to maximise uptake and long-term use. LvL UP includes a motivational interviewing and storytelling approach to deliver the coaching sessions, as well as progress feedback and gamification. Offline materials are also offered to allow users access to essential intervention content without needing a mobile device.ConclusionsThe development process of LvL UP 1.0 led to an evidence-based and user-informed smartphone-based intervention aimed at preventing NCDs and CMDs. LvL UP is designed to be a scalable, engaging, prevention-oriented, holistic intervention for adults at risk of NCDs and CMDs. A feasibility study, and subsequent optimisation and randomised-controlled trials are planned to further refine the intervention and establish effectiveness. The development process described here may prove helpful to other intervention developers.

Background Mental health interventions delivered through mobile health (mHealth) technologies can increase the access to mental health services, especially among university students. The development of mHealth intervention is complex and needs to be context sensitive. There is currently limited evidence on the perceptions, needs, and barriers related to these interventions in the Southeast Asian context. Objective This qualitative study aimed to explore the perception of university students and mental health supporters in Singapore about mental health services, campaigns, and mHealth interventions with a focus on conversational agent interventions for the prevention of common mental disorders such as anxiety and depression. Methods We conducted 6 web-based focus group discussions with 30 university students and one-to-one web-based interviews with 11 mental health supporters consisting of faculty members tasked with student pastoral care, a mental health first aider, counselors, psychologists, a clinical psychologist, and a psychiatrist. The qualitative analysis followed a reflexive thematic analysis framework. Results The following 6 main themes were identified: a healthy lifestyle as students, access to mental health services, the role of mental health promotion campaigns, preferred mHealth engagement features, factors that influence the adoption of mHealth interventions, and cultural relevance of mHealth interventions. The interpretation of our findings shows that students were reluctant to use mental health services because of the fear of stigma and a possible lack of confidentiality. Conclusions Study participants viewed mHealth interventions for mental health as part of a blended intervention. They also felt that future mental health mHealth interventions should be more personalized and capable of managing adverse events such as suicidal ideation.

Background Conversational agents (CAs) are increasingly used in health care to deliver behavior change interventions. Their evaluation often includes categorizing the behavior change techniques (BCTs) using a classification system of which the BCT Taxonomy v1 (BCTTv1) is one of the most common. Previous studies have presented descriptive summaries of behavior change interventions delivered by CAs, but no in-depth study reporting the use of BCTs in these interventions has been published to date. Objective This review aims to describe behavior change interventions delivered by CAs and to identify the BCTs and theories guiding their design. Methods We searched PubMed, Embase, Cochrane’s Central Register of Controlled Trials, and the first 10 pages of Google and Google Scholar in April 2021. We included primary, experimental studies evaluating a behavior change intervention delivered by a CA. BCTs coding followed the BCTTv1. Two independent reviewers selected the studies and extracted the data. Descriptive analysis and frequent itemset mining to identify BCT clusters were performed. Results We included 47 studies reporting on mental health (n=19, 40%), chronic disorders (n=14, 30%), and lifestyle change (n=14, 30%) interventions. There were 20/47 embodied CAs (43%) and 27/47 CAs (57%) represented a female character. Most CAs were rule based (34/47, 72%). Experimental interventions included 63 BCTs, (mean 9 BCTs; range 2-21 BCTs), while comparisons included 32 BCTs (mean 2 BCTs; range 2-17 BCTs). Most interventions included BCTs 4.1 “Instruction on how to perform a behavior” (34/47, 72%), 3.3 “Social support” (emotional; 27/47, 57%), and 1.2 “Problem solving” (24/47, 51%). A total of 12/47 studies (26%) were informed by a behavior change theory, mainly the Transtheoretical Model and the Social Cognitive Theory. Studies using the same behavior change theory included different BCTs. Conclusions There is a need for the more explicit use of behavior change theories and improved reporting of BCTs in CA interventions to enhance the analysis of intervention effectiveness and improve the reproducibility of research.

BACKGROUND Rapid proliferation of mental health interventions delivered through conversational agents (CA) calls for high-quality evidence to support their implementation and adoption. The choice of outcomes, outcome measurement instruments and assessment modalities play an important role in ensuring the relevant and high-quality evaluation of such interventions. OBJECTIVE We aimed to identify the types of outcomes, outcome measurement instruments, and assessment modalities used to assess the clinical, user experience, and technical outcomes in studies that evaluated the effectiveness of CA interventions for mental health. METHODS We undertook a scoping review of the relevant literature to review the types of outcomes, outcome measurement instruments, and the assessment platform in studies that evaluated the effectiveness of CA mental health interventions. We performed a comprehensive search of electronic databases up to April 2021. RESULTS We included 32 studies that targeted the promotion of mental well-being (17/32, 53%) and the treatment and monitoring of mental health symptoms (15/32, 66%). The studies reported 203 outcome measurement instruments used to measure clinical (123/203, 61%), user experience outcomes (75/203, 37%), technical outcomes (2/203, 1%), and other outcomes (3/203, 1%). Most of the outcome measurement instruments were used only in one study (150/203, 74%), were self-reported questionnaires (170/203, 84%) and most were delivered electronically via survey platforms (61/203, 30%). No validity evidence was cited for more than half of the outcome measurement instruments (107/203, 53%), which were largely created or adapted for the study in which they were used (95/107, 89%). CONCLUSIONS The diversity of outcomes and the choice of outcome measurement instruments employed in studies on CAs for mental health points to the need for an established minimum core outcome set and greater use of validated instruments. Future studies should also capitalize on the affordances made available by the CAs and smartphones to streamline the evaluation and reduce participants’ input burden inherent to self-reporting.