Cross-validating the embedded performance validity indicators in the Rey Auditory Verbal Learning Test in mixed neuropsychiatric and attention-deficit/hyperactivity disorder clinical samples.

Abstract

Objective: The Rey Auditory Verbal Learning Test (RAVLT) is a common test of learning and memory with established embedded validity indicators (EVIs), including the Effort Score (ES) and Forced Choice (FC) recognition. Currently, the literature lacks a head-to-head comparison of the classification accuracy of ES and FC for detecting invalid test performance. This study aimed to cross-validate ES and FC in two large, diverse, independent samples. Method: This cross-sectional study included a mixed neuropsychiatric (n = 209) and attention-deficit/hyperactivity disorder (ADHD) sample (n = 544). Validity groups were established using multiple criterion performance validity tests (PVTs), resulting in 242 valid/57 invalid and 480 valid/64 invalid performances in the neuropsychiatric and ADHD samples, respectively. Results: ES and FC both demonstrated robust classification accuracy. The optimal ES cut score was significantly higher for the ADHD (≤13; 41% sensitivity/90% specificity) than the neuropsychiatric (≤3; 33% sensitivity/89% specificity) sample. Classification accuracies of FC ranged from weak in the ADHD (area under the curve [AUC] = 0.67) to excellent in the neuropsychiatric (AUC = 0.80) sample. The FC cut score was ≤12 (54% sensitivity/91% specificity) in the neuropsychiatric and ≤14 in the ADHD (41% sensitivity/93% specificity) sample. Both EVIs were significant predictors of validity status when examined in combination. Conclusions: Both the RAVLT-ES and RAVLT-FC were effective as PVTs; however, their relative utility varied by sample such that FC had stronger classification statistics in the neuropsychiatric sample (which included a higher incidence of cognitive impairment and dementia), whereas ES was more robust in the adult ADHD sample. Moreover, considering both EVIs may result in greater accuracy than isolation.

A large-scale investigation of the classification accuracy of various performance validity tests in a medical-legal setting.

“…The genuine memory impairment profile was not applied. Although methodological standards (Boone, 2013; Larrabee, 2014; Lippa, 2018; Sweet et al, 2021) as well as established practice (Jinkerson et al, 2023; Kulas et al, 2014; Larrabee, 2014; Ord et al, 2010; Robinson et al, 2023; Schroeder et al, 2019; Sharland et al, 2018; Tse et al, 2023) in PVT research dictate that criterion groups should be established using multiple measures, the WMT is a robust free-standing PVT containing multiple trials that sample the target construct over time. The final determination ( Pass/Fail ) is made based on three indicators.…”

“…Abeare, Romero, et al, 2021; Ali et al, 2023; Blavier et al, 2023; Crişan & Erdodi, 2022; Cutler et al, 2022; Denning, 2012, 2021; L. A. Erdodi, 2023b; Fazio et al, 2017; Gegner et al, 2022; Giromini et al, 2020; Green, 2007; Greve, Etherton, et al, 2009; Holcomb et al, 2023; Jones, 2013; Kim et al, 2010; Kulas et al, 2014; Larrabee, 2005; Lichtenstein et al, 2017; Messa et al, 2022; Rai et al, 2019; Schroeder & Martin, 2021; Schroeder et al, 2013; Tse et al, 2023; Tyson et al, 2023; Webber et al, 2018; Wisdom et al, 2012).…”

“…Most free-standing PVTs employ a forced-choice recognition paradigm (e.g., Allen et al, 1997;Binder, 1993;Green, 2003;Pearson, 2009;Tombaugh, 1996), as it has been repeatedly shown to be a powerful psychometric model for differentiating credible from noncredible responding (K. Abeare, Romero, et al, 2021;Ali et al, 2023;Blavier et al, 2023;Crişan & Erdodi, 2022;Cutler et al, 2022;Denning, 2012Denning, , 2021L. A. Erdodi, 2023b;Fazio et al, 2017;Gegner et al, 2022;Giromini et al, 2020;Green, 2007;Greve, Etherton, et al, 2009;Holcomb et al, 2023;Jones, 2013;Kim et al, 2010;Kulas et al, 2014;Larrabee, 2005;Lichtenstein et al, 2017;Messa et al, 2022;Rai et al, 2019;Schroeder et al, 2013;Tse et al, 2023;Tyson et al, 2023;Webber et al, 2018;Wisdom et al, 2012).…”

Distinguishing genuine from simulated dissociative identity disorder using the Morel Emotional Numbing Test.

“…Yet, many DID studies have focused on embedded SVTs (e.g., Brand & Chasson, 2015; Brand et al, 2014). To improve specificity in embedded measures, researchers will often propose population-specific cutoffs (e.g., Erdodi et al, 2014; Finley, Cerny, et al, 2024; Tse et al, 2023). However, few studies have proposed cutoffs for DID (Barth et al, 2023).…”

From “below chance” to “a single error is one too many”: Evaluating various thresholds for invalid performance on two forced choice recognition tests

“…This a low bar that hardly provides evidence for a valid response set. Indeed, empirical investigations converged on the conclusion that the optimal cutoffs on FCR-based PVTs are invariably well above chance level (Holcomb, Roth, et al, 2022;Jones, 2013;Resch et al, 2021Resch et al, , 2022Tse et al, 2023).…”