Evaluating cognitive outcome measures for MS clinical trials: what is a clinically meaningful change?

Abstract

Brief cognitive assessments are increasingly emphasized in MS treatment studies and clinical care. While much is known about the reliability of several widely-used neuropsychological tests, interpretation of the changes in individual patients is inadequate. The FDA offers guidance on the issue, as related to patient-reported outcomes. Unfortunately, cognitive ability is only weakly correlated with the frequency and severity of self-reported cognitive problems. In this review, we critically examined the psychometrics of neuropsychological testing in MS, emphasizing statistical and anchor-based approaches to interpreting clinically meaningful change. We suggest that there are two paths forward that should be currently pursued. First, to employ co-primary outcomes, including a brief cognitive test and a clinician or observer’s impression on a scale of change, where successful treatment would require showing significant improvement in both measures. Secondly, to work toward showing that when reliable brief cognitive tests are employed, increments of statistically-relevant change would correlate with changes in clinically-relevant anchors (such as vocational disability or clinical relapses with cognitive impairment). The latter goal will allow a more parsimonious and scientifically efficient approach of utilizing only the brief cognitive test as a primary outcome. While some progress has been made in this direction, more research is needed. We are of the opinion that data from both the statistical and clinically meaningful approaches will be necessary to develop valid definitions of meaningful change on cognitive outcome measures, and that it would be best to pursue research using tests that already have well-established reliability and validity.

Validity of the Symbol Digit Modalities Test as a cognition performance outcome measure for multiple sclerosis

“…As noted in a prior topical review, 112 cognitive performance is not reliably correlated with patient self-reports of cognitive symptoms. Patient self-reports of cognitive functioning (cognition patient-reported outcomes (PROs)) are less related to actual cognitive performance than to other subjective symptoms such as depression and fatigue.…”

Are cognitive interventions for Multiple Sclerosis effective and feasible?

“…2) Statistical significance is typically overvalued and few acknowledge the importance of a clinically meaningful change that may result from cognitive intervention. Benedict and Walton (2012) have recently drawn attention to this issue, referring to the use of patient-reported outcomes (PROs) and of clinically meaningful change in MS studies. Namely, Food and Drug Administration (FDA) Guidance highlights that, besides providing statistical evidence of a given treatment, one should also examine the presence of individual meaningful change (Benedict and Walton, 2012).…”

Cognitive functions over the course of 1 year in multiple sclerosis patients treated with disease modifying therapies

“…Moreover, to measure cognitive change, repeated testing is required. This is challenging, because several factors may confound the assessment of true cognitive change, such as random variations, variation due to daily fluctuations of mood, motivation or alertness of patients, measurement errors, test characteristics, the test-retest reliability of a measure or the statistical phenomenon of regression towards the mean [Heaton et al 2001;Benedict and Walton, 2012;Till et al 2013]. Furthermore, practice effects may obscure the assessment of clinically reliable change.…”