Validity Evidence for the Use of Automated Neuropsychologic Assessment Metrics As a Screening Tool for Cognitive Impairment in Systemic Lupus Erythematosus

Abstract

Objective. Screening for cognitive impairment in systemic lupus erythematosus (SLE) conventionally relies on the American College of Rheumatology (ACR) neuropsychologic battery (NB), which is not universally available. To develop a more accessible screening approach, we assessed validity of the Automated Neuropsychological Assessment Metrics (ANAM). Using the ACR NB as the gold standard for cognitive impairment classification, the objectives were 1) to measure overall discriminative validity of the ANAM for cognitive impairment versus no cognitive impairment, 2) to identify ANAM subtests and scores that best differentiate patients with cognitive impairment from those with no cognitive impairment, and 3) to derive ANAM composite indices and cutoffs. Methods. A total of 211 consecutive adult patients, female and male, with SLE were administered the ANAM and ACR NB. 1) For overall discriminative validity of the ANAM, we compared patients with cognitive impairment versus those with no cognitive impairment on 4 scores. 2) Six ANAM models using different scores were developed, and the most discriminatory subtests were selected using logistic regression analyses. The area under the receiver operating characteristic curve (AUC) was calculated to establish ANAM validity against the ACR NB. 3) ANAM composite indices and cutoffs were derived for the best models, and sensitivities and specificities were calculated. Results. Patients with no cognitive impairment performed better on most ANAM subtests, supporting ANAM's discriminative validity. Cognitive impairment could be accurately identified by selected ANAM subtests with top models, demonstrating excellent AUCs of 81% and 84%. Derived composite indices and cutoffs demonstrated sensitivity of 78-80% and specificity of 70%. Conclusion. This study provides support for ANAM's discriminative validity for cognitive impairment and utility for cognitive screening in adult SLE. Derived composite indices and cutoffs enhance clinical applicability.

Measures of Cognition in Rheumatic Diseases

“…A total cognitive function score (a summation of all test scores indicating impairment) of 4 or more indicated CI (5). Another study in SLE defined CI as a Z score of −1.5 or less in two or more domains, defined cognitively normal as Z scores in all domains greater than −1.5, and defined indeterminate as a Z score of −1.5 or less in only one domain (10).…”

“…Studies within rheumatology have also adopted the ACR‐NB and have made appropriate adjustments. For instance, Tayer‐Shifman et al substituted the CVLT with the HVLT‐R (10) because the HVLT‐R is shorter, and Shin et al modified the ACR‐NB for use in RA to control for the effects of hand–motor dysfunction (5). The tests included in the ACR‐NB are general neuropsychological tests designed to detect deficits in specific cognitive domains and are widely used in many different clinical populations, such as geriatrics, traumatic brain injury, etc.…”

“…Studies have documented test‐retest reliability for many individual ANAM subtests across varying retest intervals. Most published estimates for ANAM represent alternate forms reliability because the ANAM test system automatically detects when an individual has been previously assessed and will iterate a new stimulus set unless otherwise specified (10). Individual ANAM measures show moderate to good test‐retest reliability (0.40‐0.79) and internal consistency in most studies (44‐48).…”

“…Selected ANAM subtests and scores (coding immediate recall throughput, spatial processing reaction time, matching to sample accuracy, and continuous performance lapses) were shown by multiple regression analyses to predict more than half the variance in the traditional neuropsychological test batteries. Tayer‐Shifman et al added construct validity evidence of the ANAM GNS battery when compared with the ACR‐NB for assessing cognitive function in 211 patients with SLE (10). Using receiver operating characteristic (ROC) curve analysis, they demonstrated that specific subtests are associated with CI in patients with SLE (code substitution learning, code substitution delayed, spatial processing, tapping left hand, simple reaction time repeated, go/no‐go, tower puzzle, and two‐choice reaction time), with an excellent area under the curve (AUC) of 84% (95% confidence interval: 0.84‐0.87), sensitivity of 76%, and specificity of 80% (10).…”

“…The neuropsychological battery remains the gold standard in diagnosing CI across many, if not all, clinical rheumatological populations. However, there have been several alternatives to the standard neuropsychological battery that have been validated and have shown clinical utility in the rheumatic population (particularly in SLE), including various screening tests, standalone neuropsychological tests (eg, Hopkins Verbal Learning Test Revised [HVLT‐R], Controlled Oral Word Association Test [COWAT]), and computerized batteries (eg, the Automated Neuropsychological Assessment Metrics [ANAM] or others) (2,10‐12). The use of these alternatives may be due to the length of time required for testing, the need for trained personnel to administer the battery and interpret scores, and the costs associated with specific neuropsychological tests.…”