Symbol Digit Modalities Test Variant in a Smartphone App for Persons With Multiple Sclerosis: Validation Study

Abstract

Background The decline of cognitive processing speed (CPS) is a common dysfunction in persons with multiple sclerosis (MS). The Symbol Digit Modalities Test (SDMT) is widely used to formally quantify CPS. We implemented a variant of the SDMT in MS sherpa, a smartphone app for persons with MS. Objective The aim of this study was to investigate the construct validity and test-retest reliability of the MS sherpa smartphone variant of the SDMT (sSDMT). Methods We performed a validation study with 25 persons with relapsing-remitting MS and 79 healthy control (HC) subjects. In the HC group, 21 subjects were matched to the persons with MS with regard to age, gender, and education and they followed the same assessment schedule as the persons with MS (the “HC matched” group) and 58 subjects had a less intense assessment schedule to determine reference values (the “HC normative” group). Intraclass correlation coefficients (ICCs) were determined between the paper-and-pencil SDMT and its smartphone variant (sSDMT) on 2 occasions, 4 weeks apart. Other ICCs were determined for test-retest reliability, which were derived from 10 smartphone tests per study participant, with 3 days in between each test. Seven study participants with MS were interviewed regarding their experiences with the sSDMT. Results The SDMT scores were on average 12.06% higher than the sSDMT scores, with a standard deviation of 10.68%. An ICC of 0.838 was found for the construct validity of the sSDMT in the combined analysis of persons with MS and HC subjects. Average ICCs for test-retest reliability of the sSDMT for persons with MS, the HC matched group, and the HC normative group were 0.874, 0.857, and 0.867, respectively. The practice effect was significant between the first and the second test of the persons with MS and the HC matched group and trivial for all other test-retests. The interviewed study participants expressed a positive attitude toward the sSDMT, but they also discussed the importance of adapting a smartphone cognition test in accordance with the needs of the individual persons with MS. Conclusions The high correlation between sSDMT and the conventional SDMT scores indicates a very good construct validity. Similarly, high correlations underpin a very good test-retest reliability of the sSDMT. We conclude that the sSDMT has the potential to be used as a tool to monitor CPS in persons with MS, both in clinical studies and in clinical practice.

A smartphone sensor-based digital outcome assessment of multiple sclerosis

“…The Spearman rho of 0.82 is comparable to the previously reported correlation between other smartphone-based versions of the SDMT and the pen-and-paper version of the SDMT (r = 0.71-0.85). 11,12 In our study, we noted that the e-SDMT scores tended to be lower than the oral SDMT scores in PwMS and HC. This is likely due to the longer time required to select the correct response on a smartphone display compared with saying the correct response out loud.…”

“…Test-retest reliability was consistent with ICCs reported for standard clinical measures in PwMS. 22 As 11,12 upper extremity function, 13 and gait domains, 10 despite differences in test design and features. The active tests and passive monitoring also showed mostly fair correlations with EDSS and respective MSIS-29 subscales and items, indicating that measurements obtained with the Floodlight PoC app agree both with the overall level of MS-related disability and participant's perception of the impact of their disease.…”

“…7 They are also rater-independent, offering more objective measures of functional ability. [8][9][10] In recent years, sensor-based tests have been increasingly used to assess functional ability in MS, [7][8][9][10][11][12][13][14] as well as in Parkinson's disease 15 and Huntington's disease. 16,17 The Floodlight Proof-of-Concept (PoC) app was designed to remotely measure, at home in an unsupervised setting, functional ability in cognitive, upper extremity, and gait and balance domains using a smartphone device without additional hardware.…”

Electronic Health Interventions in the Case of Multiple Sclerosis: From Theory to Practice

“…It prepares caregivers for the special needs of MS and empowers patients to care for themselves by asking questions and providing guidelines on the website [ 32 ]. Self-managing MS is feasible with MS Energize (AUT Ventures, New Zealand; New Zealand; Kiwinet, New Zealand; MEA Mobile, Stuttgart, Germany; 2019) [ 33 ], MSCopilot (AD SCIENTIAM, Paris, France; 2019) [ 34 ], MS COMPASS++ (PEARS HEALTH CYBER, Czech Republic; 2015) [ 35 ], via digital diary [ 36 ], MS Invigor8 [ 37 ], Managing Fatigue [ 38 ], MS Sherpa (MS sherpa BV, Nijmegen, Netherlands; 2019) [ 39 ], MyMS&Me (Irody, Inc., Boston, MA, USA; 2020) [ 40 ] and other mobile applications [ 41 ]. Especially wireless and wearable devices are useful interventions to enhance rehabilitation in pwMS [ 26 , 42 ].…”

Reliability, construct and concurrent validity of a smartphone-based cognition test in multiple sclerosis

“…Assessment of cognition in MS through wearable devices, such as tablets and smartphones, has been studied previously. [7][8][9][10][11][12][13] These mostly instrumented versions of the Symbol Digit Modalities Test (SDMT) could differentiate patients with MS from healthy controls (HC) and were found to be reliable and valid for measuring information processing speed. 8,13 However, digital monitoring tools for cognition are not yet employed in clinical practice due to different challenges including the lack of standardisation.…”