Cross-validation of a forced-choice validity indicator to enhance the clinical utility of the Rey Auditory Verbal Learning Test.

Abstract

A forced-choice (FC) recognition trial was recently developed as an embedded validity indicator for the Rey Auditory Verbal Learning Test (RAVLT), although it has not been replicated outside of the initial validation study. This study cross-validated the RAVLT FC trial for detecting invalid neuropsychological test performance and assessed the degree to which material-specific verbal memory impairment severity impacts its classification accuracy as a performance validity test (PVT). This cross-sectional study included 172 neuropsychiatric patients who completed the RAVLT and 4 independent criterion PVTs, which were used to classify validity groups (134 valid/38 invalid). Overall results showed the RAVLT FC had excellent classification accuracy for detecting invalid performance at a ≤13 cut-score (66% sensitivity/87% specificity). When patients were subdivided by memory impairment status, FC retained excellent classification accuracy among the normal memory and mild memory impairment groups with 66%-82% sensitivity and ≥89% specificity. Accuracy decreased among those with severe memory impairment, but remained significant with a lower, alternative cut-score of ≤11 (37% sensitivity/88% specificity). Findings were consistent with FC trials developed for other memory measures and support the utility of this novel RAVLT FC index for reliably identifying invalid performance, even in the context of significant verbal memory impairment. Public Significance StatementThe Rey Auditory Verbal Learning Test (RAVLT) is a well-validated and commonly used verbal learning/memory test in neuropsychological evaluation. This study demonstrated that a forced-choice recognition trial previously developed as an embedded performance validity test greatly enhances the clinical and research utility of the RAVLT as it accurately detects invalid test performance, even in the context of genuine memory impairment of increasing severity, without contributing to significantly greater evaluation time/burden.

Cross-validating the Dot Counting Test Among an Adult ADHD Clinical Sample and Analyzing the Effect of ADHD Subtype and Comorbid Psychopathology

“…Finally, results revealed remarkably stable classification accuracy, optimal cut-scores, and sensitivity/specificity values regardless of whether those with a single PVT failure were retained or removed from the valid group. This finding is consistent with several recent studies (e.g., McCaul et al, 2018;Rhoads, Neale, et al, 2021;Soble et al, 2021) showing no differences if those with one PVT fail are retained, even in samples with a high base rate of external incentive (e.g., Resch et al, 2021). In addition, this finding supports the inclusion of those with one embedded PVT failure as valid in PVT cross-validation studies using clinical samples, given that their inclusion reflects actual clinical practice, does not reduce sample sizes, and does not artificially deflate neurocognitive test results or inflate PVT classification accuracy statistics, regardless of the presence of external incentive (Jennette et al, 2021).…”

Cross-validating the embedded performance validity indicators in the Rey Auditory Verbal Learning Test in mixed neuropsychiatric and attention-deficit/hyperactivity disorder clinical samples.

“…In contrast, as replicated in other studies (e.g., Pliskin et al, 2021; Whitney & Davis, 2015), patients with bona fide cognitive impairment required substantially lower ES cutoffs. Our findings are also largely consistent with recent research indicating that FC is more robust to dementia and/or profound cognitive impairment than ES (Poreh et al, 2016; Soble, Sharp, et al, 2021). Indeed, cutoff scores of approximately ≤12 may be ideal for patients with mild dementia, as originally suggested by Poreh and colleagues (2016).…”

Comprehensive Analysis of MMPI-2-RF Symptom Validity Scales and Performance Validity Test Relationships in a Diverse Mixed Neuropsychiatric Setting

“…This battery included the Test of Premorbid Function (TOPF; Pearson, 2009), Verbal Fluency (F/A/S and Animal Naming; Heaton et al, 2004), Rey Auditory Verbal Learning Test (RAVLT; Schmidt, 1996), Brief Visuospatial Memory Test-Revised (BVMT-R; Benedict, 1997), Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV; Wechsler, 2008) Processing Speed Intext (PSI), Trail Making Test (TMT; Heaton et al, 2004), and Stroop Color and Word Test (Golden, 1978). Of note, FSIQ 100.18 (9.20) 77-128 100.83 (8.82) 77-128 97.51 (10.46) 79-115 3 although embedded PVTs derived from the RAVLT (Boone et al, 2005;Pliskin et al, 2021;Soble et al, 2021a) and BVMT-R (Bailey et al, 2018;Resch et al, 2022b), Verbal Fluency, TMT, and Stroop (Khan et al, 2022;White et al, 2020) have been identified; these embedded indicators were not included in the reference standard in order to avoid criterion contamination by keeping the neurocognitive tests fully independent from the criterion PVTs used to establish the validity groups. Among the 30 total patients who were actively compensation-seeking at the time of evaluation (see Table 1), 27% (8/30) obtained ≥ 2 PVT failures and were classified into the invalid group, whereas the remaining 73% (22/30) of those who were actively compensationseeking demonstrated valid test performance (i.e., ≤ 1 PVT failure).…”