Mobile cognitive testing in patients with schizophrenia: A controlled study of feasibility and validity

Dupuy, Maud; Misdrahi, D.; N’Kaoua, Bernard; Tessier, A.; Bouvard, Alexandra; Schweitzer, Pierre; Auriacombe, Marc; Serre, Fuschia; Fatséas, Mélina; Swendsen, Joël

doi:10.1016/j.jtcc.2018.02.002

Cited by 16 publications

(16 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Results of Brouillette et al (2013) supported the test–retest reliability in one MCT measuring processing speed, and Timmers et al (2014) found high test–retest correlations in an MCT assessing short term memory. Other investigations have reported good convergent and discriminant validity of MCTs compared to laboratory tests, with more shared variance between tests of similar constructs than between tests of distinct constructs ( Moore et al, 2017a , 2020 ; Dupuy et al, 2018 ; Sliwinski et al, 2018 ). However, no studies to our knowledge have reported on estimates of reliable change in MCTs, and this is a limitation of the current literature, given that a major advantage of MCTs is their brevity and potential for repeatability ( Allard et al, 2014 ; Sliwinski et al, 2018 ).…”

Section: Discussionmentioning

confidence: 85%

Test–Retest Reliability and Reliable Change Estimates for Four Mobile Cognitive Tests Administered Virtually in Community-Dwelling Adults

Patten

Muzeau²,

VanRavenhorst-Bell

2021

Front. Psychol.

View full text Add to dashboard Cite

Objective: Remote mobile cognitive testing (MCT) is an expanding area of research, but psychometric data supporting these measures are limited. We provide preliminary data on test–retest reliability and reliable change estimates in four MCTs from SWAY Medical, Inc.Methods: Fifty-five adults from the U.S. Midwest completed the MCTs remotely on their personal mobile devices once per week for 3 consecutive weeks, while being supervised with a video-based virtual connection. The cognitive assessment measured simple reaction time (“Reaction Time”), go/no-go response inhibition (“Impulse Control”), timed visual processing (“Inspection Time”), and working memory (“Working Memory”). For each cognitive test except Working Memory, we analyzed both millisecond (ms) responses and an overall SWAY composite score.Results: The mean age of the sample was 26.69years (SD=9.89; range=18–58). Of the 55 adults, 38 (69.1%) were women and 49 (89.1%) used an iPhone. Friedman’s ANOVAs examining differences across testing sessions were nonsignificant (ps>0.31). Intraclass correlations for Weeks 1–3 were: Reaction Time (ms): 0.83, Reaction Time (SWAY): 0.83, Impulse Control (ms): 0.68, Impulse Control (SWAY): 0.80, Inspection Time (ms): 0.75, Inspection Time (SWAY): 0.75, and Working Memory (SWAY): 0.88. Intraclass correlations for Weeks 1–2 were: Reaction Time (ms): 0.75, Reaction Time (SWAY): 0.74, Impulse Control (ms): 0.60, Impulse Control (SWAY): 0.76, Inspection Time (ms): 0.79, Inspection Time (SWAY): 0.79, and Working Memory (SWAY): 0.83. Natural distributions of difference scores were calculated and reliable change estimates are presented for 70, 80, and 90% CIs.Conclusion: Test–retest reliability was adequate or better for the MCTs in this virtual remote testing study. Reliable change estimates allow for the determination of whether a particular level of improvement or decline in performance is within the range of probable measurement error. Additional reliability and validity data are needed in other age groups.

show abstract

Section: Discussionmentioning

confidence: 85%

Test–Retest Reliability and Reliable Change Estimates for Four Mobile Cognitive Tests Administered Virtually in Community-Dwelling Adults

Patten

Muzeau²,

VanRavenhorst-Bell

2021

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Regarding user experience, both structured and informal user feedback collection has been iterative and ongoing throughout the development process. The MindLogger apps described in this paper use an assessment frequency, duration, and question content that has been substantially validated over recent years by members of our team for a wide range of age groups (children, adults, and older adults) [ 62 - 66 ], for healthy individuals [ 62 , 63 , 67 ], and for persons with diverse forms of mental [ 62 , 63 , 67 - 71 ] or physical [ 72 - 74 ] disorder. The NIMH’s predecessor to the MindLogger applet described below (using a highly similar protocol for daily assessments and with identical question content) documented an average completion rate of 77.9% (SE 0.81%) for repeated daily assessments, with no significant fatigue effect, defined as an increase in missing data as a function of time in the study (ordinary least squares linear model β coefficient=–.041; P <.001).…”

Section: Methodsmentioning

confidence: 99%

Remote Digital Psychiatry for Mobile Mental Health Assessment and Therapy: MindLogger Platform Development Study

Klein¹,

Clucas²,

Krishnakumar³

et al. 2021

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Background Universal access to assessment and treatment of mental health and learning disorders remains a significant and unmet need. There are many people without access to care because of economic, geographic, and cultural barriers, as well as the limited availability of clinical experts who could help advance our understanding and treatment of mental health. Objective This study aims to create an open, configurable software platform to build clinical measures, mobile assessments, tasks, and interventions without programming expertise. Specifically, our primary requirements include an administrator interface for creating and scheduling recurring and customized questionnaires where end users receive and respond to scheduled notifications via an iOS or Android app on a mobile device. Such a platform would help relieve overwhelmed health systems and empower remote and disadvantaged subgroups in need of accurate and effective information, assessment, and care. This platform has the potential to advance scientific research by supporting the collection of data with instruments tailored to specific scientific questions from large, distributed, and diverse populations. Methods We searched for products that satisfy these requirements. We designed and developed a new software platform called MindLogger, which exceeds the requirements. To demonstrate the platform’s configurability, we built multiple applets (collections of activities) within the MindLogger mobile app and deployed several of them, including a comprehensive set of assessments underway in a large-scale, longitudinal mental health study. Results Of the hundreds of products we researched, we found 10 that met our primary requirements with 4 that support end-to-end encryption, 2 that enable restricted access to individual users’ data, 1 that provides open-source software, and none that satisfy all three. We compared features related to information presentation and data capture capabilities; privacy and security; and access to the product, code, and data. We successfully built MindLogger mobile and web applications, as well as web browser–based tools for building and editing new applets and for administering them to end users. MindLogger has end-to-end encryption, enables restricted access, is open source, and supports a variety of data collection features. One applet is currently collecting data from children and adolescents in our mental health study, and other applets are in different stages of testing and deployment for use in clinical and research settings. Conclusions We demonstrated the flexibility and applicability of the MindLogger platform through its deployment in a large-scale, longitudinal, mobile mental health study and by building a variety of other mental health–related applets. With this release, we encourage a broad range of users to apply the MindLogger platform to create and test applets to advance health care and scientific research. We hope that increasing the availability of applets designed to assess and administer interventions will facilitate access to health care in the general population.

show abstract

“…Compliance was high, fatigue effects were not observed, and practice effects occurred as a function of study duration, but this relationship was observed for both the patient and control groups. Outcomes for the high-frequency abbreviated assessments correlated considerably with the outcomes from the validated full-length assessments in both patients and controls, demonstrating convergent validity for the high-frequency assessments [ 76 ]. However, the full-length assessment, against which the abbreviated assessment is validated, does not necessarily need to be administered at a testing facility.…”

Section: How To Validate High-frequency Assessments For Remote Researchmentioning

confidence: 99%

“…In this case study, it was feasible to administer the full-length assessments daily in natural settings. However, many full-length assessments might be too long to administer as frequently as once a day [ 78 ] or need to be administered at a testing facility [ 76 ] (eg, when specialist equipment or a trained administrator is required). The results in this case study demonstrate how extraneous factors (eg, the referendum) can affect outcomes.…”

Section: How To Validate High-frequency Assessments For Remote Researchmentioning

confidence: 99%

Developing Digital Tools for Remote Clinical Research: How to Evaluate the Validity and Practicality of Active Assessments in Field Settings

Ferrar¹,

Griffith²,

Skirrow³

et al. 2021

J Med Internet Res

View full text Add to dashboard Cite

The ability of remote research tools to collect granular, high-frequency data on symptoms and digital biomarkers is an important strength because it circumvents many limitations of traditional clinical trials and improves the ability to capture clinically relevant data. This approach allows researchers to capture more robust baselines and derive novel phenotypes for improved precision in diagnosis and accuracy in outcomes. The process for developing these tools however is complex because data need to be collected at a frequency that is meaningful but not burdensome for the participant or patient. Furthermore, traditional techniques, which rely on fixed conditions to validate assessments, may be inappropriate for validating tools that are designed to capture data under flexible conditions. This paper discusses the process for determining whether a digital assessment is suitable for remote research and offers suggestions on how to validate these novel tools.

show abstract

Mobile cognitive testing in patients with schizophrenia: A controlled study of feasibility and validity

Cited by 16 publications

References 71 publications

Test–Retest Reliability and Reliable Change Estimates for Four Mobile Cognitive Tests Administered Virtually in Community-Dwelling Adults

Test–Retest Reliability and Reliable Change Estimates for Four Mobile Cognitive Tests Administered Virtually in Community-Dwelling Adults

Remote Digital Psychiatry for Mobile Mental Health Assessment and Therapy: MindLogger Platform Development Study

Developing Digital Tools for Remote Clinical Research: How to Evaluate the Validity and Practicality of Active Assessments in Field Settings

Contact Info

Product

Resources

About