A large-scale investigation of the classification accuracy of various performance validity tests in a medical-legal setting.

Abstract

Objective: This study aimed to investigate the classification accuracy of various free-standing and embedded performance validity tests (PVTs) administered in a medical-legal setting. Method: Archival data were analyzed from 4,721 adults (M Age = 42.3 years; M Education = 12.0) referred for psychological or neuropsychological assessments in a medical-legal or civil disability context. Classification accuracy was compared at multiple cutoffs on nine PVTs of interest (three free-standing and six embedded), using the Word Memory Test as criterion. Results: Most previously published PVT cutoffs achieved acceptable specificity. Freestanding PVTs demonstrated better classification accuracy than embedded validity indicators. Conclusions: Results support the utility of commonly used free-standing and embedded PVTs with medical-legal or forensic civil disability samples. The findings highlight the importance of combining multiple PVTs to improve overall classification accuracy. Public Significance StatementThe utility of performance validity tests is typically evaluated in relatively small samples (n: 100-150), and results tend to fluctuate across clinical characteristics and settings. Large scale investigations are needed to evaluate the generalizability of findings.

Base rates of failure on various performance validity tests as a function of age in adults referred for neuropsychological assessment.

“…Fourth, passing the WMT was an inclusion criterion with the implicit assumption that most neurocognitive profiles in the study are valid. Although the WMT is one of the most sensitive freestanding PVTs (Erdodi et al, 2024; Fazio et al, 2015; Rai et al, 2023; Tyson et al, 2023), given the unilateral emphasis on specificity, passing any PVT has limited negative predictive power (Chafetz, 2022; Mossman et al, 2018). Therefore, some of the BR Fail observed on the target PVTs may in fact represent true positives.…”

“…Inclusion criteria were age 16-69 and passing the Word Memory Test (WMT; Green, 2003) at standard cutoffs to minimize the confounding effect of neurodegenerative disorders and noncredible responding on target PVTs. Requiring participants to pass the WMT eliminates a large proportion of invalid profiles (Fazio et al, 2015;Rai et al, 2023), allowing for a more meaningful analysis of the relationship between age and BR Fail . Otherwise, age-related fluctuations in BR Fail on the PVTs of interest may have been contaminated by invalid performance.…”

“…Most participants (89.4%) were native speakers of English. Parts of the data set were used in previous publications focused on the classification accuracy of various PVTs (Erdodi, 2023a(Erdodi, , 2023cErdodi et al, 2019;Erdodi & Rai, 2017;Rai et al, 2023;Sabelli et al, 2021).…”

“…MSVT = Medical Symptom Validity Test (Green, 2004); STN = standard cutoffs specified in the test manual; NV-MSVT = Nonverbal Medical Symptom Validity Test (Green, 2008); TOMM = Test of Memory Malingering (Tombaugh, 1996); 1 = first trial (Denning, 2012;Jones, 2013;Kulas et al, 2014;Martin et al, 2020;Rai & Erdodi, 2021); 2 = second trial; RDS = Reliable Digit Span (Greiffenstein et al, 1994); WCST-FMS = Failure to Maintain Set on the Wisconsin Card Sorting Test (Heaton et al, 1993) criterion with the implicit assumption that most neurocognitive profiles in the study are valid. Although the WMT is one of the most sensitive freestanding PVTs (Erdodi et al, 2024;Fazio et al, 2015;Rai et al, 2023;Tyson et al, 2023), given the unilateral emphasis on specificity, passing any PVT has limited negative predictive power (Chafetz, 2022;Mossman et al, 2018). Therefore, some of the BR Fail observed on the target PVTs may in fact represent true positives.…”

Assessing the credibility of clinical presentations using performance and symptom validity tests: Current trends and future directions—Part II.

“…The articles included in the second special issue of Psychology & Neuroscience devoted this year to performance and symptom validity testing offer some new, interesting insights. The first article in this issue, by Rai et al (2023), provides classification accuracy for a wide range of cutoffs on commonly used free-standing and embedded PVTs in a very large (n = 4,721) mixed clinical/ medicolegal sample. In addition, the authors of this first article also reported very strong correlations between the base rate of failure and sensitivity (.95) and specificity (−.80), suggesting that the prevalence of noncredible presentation (operationalized as failing a given cutoff) is an influential, yet often neglected parameter in PVT research.…”