Validation of a rapid remote digital test for impaired cognition using clinical dementia rating and mini-mental state examination: An observational research study

Abstract

BackgroundThe Clinical Dementia Rating (CDR) and Mini-Mental State Examination (MMSE) are useful screening tools for mild cognitive impairment (MCI). However, these tests require qualified in-person supervision and the CDR can take up to 60 min to complete. We developed a digital cognitive screening test (M-CogScore) that can be completed remotely in under 5 min without supervision. We set out to validate M-CogScore in head-to-head comparisons with CDR and MMSE.MethodsTo ascertain the validity of the M-CogScore, we enrolled participants as healthy controls or impaired cognition, matched for age, sex, and education. Participants completed the 30-item paper MMSE Second Edition Standard Version (MMSE-2), paper CDR, and smartphone-based M-CogScore. The digital M-CogScore test is based on time-normalised scores from smartphone-adapted Stroop (M-Stroop), digit-symbols (M-Symbols), and delayed recall tests (M-Memory). We used Spearman's correlation coefficient to determine the convergent validity between M-CogScore and the 30-item MMSE-2, and non-parametric tests to determine its discriminative validity with a CDR label of normal (CDR 0) or impaired cognition (CDR 0.5 or 1). M-CogScore was further compared to MMSE-2 using area under the receiver operating characteristic curves (AUC) with corresponding optimal cut-offs.Results72 participants completed all three tests. The M-CogScore correlated with both MMSE-2 (rho = 0.54, p < 0.0001) and impaired cognition on CDR (Mann Whitney U = 187, p < 0.001). M-CogScore achieved an AUC of 0.85 (95% bootstrapped CI [0.80, 0.91]), when differentiating between normal and impaired cognition, compared to an AUC of 0.78 [0.72, 0.84] for MMSE-2 (p = 0.21).ConclusionDigital screening tests such as M-CogScore are desirable to aid in rapid and remote clinical cognitive evaluations. M-CogScore was significantly correlated with established cognitive tests, including CDR and MMSE-2. M-CogScore can be taken remotely without supervision, is automatically scored, has less of a ceiling effect than the MMSE-2, and takes significantly less time to complete.

Subjective brain fog: a four-dimensional characterization in 25,796 participants

“…The data was acquired from a smartphone application named Mindstep which contains questionnaires that gather background medical information ( Rifkin-Zybutz et al, 2021 ), evaluate cognitive function ( Alim-Marvasti et al, 2022a , b ), evaluate severity of anxiety, depression ( Kuleindiren et al, 2022 ), and post-concussive symptoms ( Alim-Marvasti et al, 2022a , b ). An application update, released in September 2022, routinely asked whether users experienced self-defined “brain fog.” The application was available to iPhone users in the Apple Application Store and has been previously validated ( Rifkin-Zybutz et al, 2021 ).…”

“…Participants gave informed consent for the collection and use of their anonymised data for research, as previously described ( Rifkin-Zybutz et al, 2021 ; Kuleindiren et al, 2022 ; Alim-Marvasti et al, 2022a , b ).…”

“…Other routinely collected data by the application that are not the focus of this study and therefore were not analysed, have been previously described ( Rifkin-Zybutz et al, 2021 ; Kuleindiren et al, 2022 ). The application routinely collected data on participant’s age, sex, presence of comorbidities, severity of anxiety and depression symptoms ( Kuleindiren et al, 2022 ), lifestyle habits including diet and self-assessed sleep quality, and evaluated cognitive scores ( Alim-Marvasti et al, 2022a , b ).…”

“…Cognitive scores were evaluated using the smartphone-based M-CogScore which combines measures of delayed recall, Stroop, and digit symbols modality test with speed of test completion ( Alim-Marvasti et al, 2022a , b ). As the exact tests have been described in detail previously, we briefly summarise the tests here.…”

“…Further details on the tests and aspect of cognition that they measure can be found in our original paper ( Alim-Marvasti et al, 2022a , b ). The only difference from the methodology described previously to validate this score was that the scores were Z-scored on this larger cohort of 25,796 individuals.…”