Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical)

Abstract

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs ( Pass, Borderline, and Fail).

Critical item (CR) analysis expands the classification accuracy of performance validity tests based on the forced choice paradigm—Replicating previously introduced CR cutoffs within the Word Choice Test.

“…Failing more conservative cutoffs was coded as 2 and 3 (strong evidence of invalid performance). The value of the EI-5 reflects the sum of the recoded components, capturing both the number and extent of PVT failures (Erdodi, 2021(Erdodi, , 2022, recognizing that the underlying construct is a continuous variable. As such, it ranges from 0 (the examinee passed all five PVTs at the most liberal cutoff) to 15 (the examinee failed all five PVTs at the most conservative cutoff).…”

“…The EI-5 is based on five embedded PVTs recoded onto a 4-point ordinal scale (0-3), following the methodology described by Erdodi (2019Erdodi ( , 2021Erdodi ( , 2022, along published cutoffs (Supplemental Table A). Passing the liberal cutoff was coded as 0 (incontrovertible Pass).…”

Examining the effect of repeat administration, alternate versions, and performance validity on letter fluency tests in a mixed clinical sample.

“…Table 1 provides a list of cutoffs and associated failure rates. As such, the EI-5s capture both the number and extent of PVT failure, capturing the dual nature (continuous and discrete) of performance validity (Erdodi, 2022). An EI-5 value of ≤1 is considered an overall Pass , whereas a value ≥4 is considered an overall Fail .…”

“…This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Denning, 2012;Erdodi, 2022;Jones, 2013;Kulas et al, 2014;Rai & Erdodi, 2021); Warrington's Recognition Memory Test-Words (Fail defined as accuracy ≤42; or time-to-completion ≥207 s; Erdodi et al, 2014;Kim et al, 2010;Zuccato et al, 2018); Word Choice Test (Fail defined as accuracy ≤45; or time-to-completion ≥171 s; Cutler, Greenacre, et al, 2022;Erdodi, 2021;Holcomb et al, 2022;Zuccato et al, 2018); FS-PVT-4 ≤ 1 was defined as valid; FS-PVT-4 ≥ 2 was defined as invalid; EI-5 VER = Erdodi Index Five-verbal (Fail defined as ≥4); EI-5 PSP = Erdodi Index Fiveprocessing speed (Fail defined as ≥4); T = demographically adjusted T scores (M = 50, SD = 10) based on norms by Heaton et al (2004); phCLS = number of phonemic clusters; d AVG = average effect size across the four scores within a given version of the letter fluency task (FAS vs. CFL); PVT = performance validity tests.…”

“…This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. Heaton et al (2004); FS-PVT-4 =number of failures on the following free-standing performance validity tests: Rey-15 Item Test free recall (Fail defined as ≤11; Ashendorf et al, 2021;Poynter et al, 2019); first trial of the Test of Memory Malingering (Fail defined as ≤43; Denning, 2012;Erdodi, 2022;Jones, 2013;Kulas et al, 2014;Rai & Erdodi, 2021); Warrington's Recognition Memory Test-Words (Fail defined as accuracy ≤42; or time-tocompletion ≥207 s; Erdodi et al, 2014;Kim et al, 2010;Zuccato et al, 2018); Word Choice Test (Fail defined as accuracy ≤45; or time-to-completion ≥171 s; Cutler, Greenacre, et al, 2022;Erdodi, 2021;Holcomb et al, 2022;Zuccato et al, 2018); FS-PVT-4 ≤ 1 was defined as valid; FS-PVT-4 ≥ 2 was defined as invalid; EI-5 VER = Erdodi Index Five-verbal (Fail defined as ≥4); EI-5 PSP = Erdodi Index Five-processing speed (Fail defined as ≥4).…”