Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

Critical item (CR) analysis expands the classification accuracy of performance validity tests based on the forced choice paradigm—Replicating previously introduced CR cutoffs within the Word Choice Test.

“…Critical item analysis was less effective at expanding sensitivity compared to the original study (2%–5% vs. 2%–17%). This modest incremental improvement is consistent with the results of previous postpublication methods to enhance the signal detection power (i.e., completion time) of the WCT (Erdodi, Tyson, et al, 2017) and its predecessor (Kim et al, 2010), or critical item analysis applied to other forced-choice recognition based PVTs (Dunn et al, 2021). However, failing critical items in addition to failing the traditional WCT cutoffs ≤47 and ≤46 increased specificity from .84 to .91 to anywhere between .92 and .93 (CR-5 ≥1) through .95–.97 (CR-7 ≥2) to 1.00 (CR-3 ≥2).…”

“…The study has a number of strengths as well. It replicated previous research on critical item analysis within the WCT from a different geographic sample, consolidating the evidence base on this method as a “signal amplifier” in PVTs (Dunn et al, 2021; Erdodi, Tyson, et al, 2018). The study demonstrated that relying on critical items as an alternative metric of performance validity can expand the WCT’s classification accuracy in both directions (i.e., increase sensitivity or specificity), although it was the most useful in containing false positive rates.…”

“…This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. ; logical memory recognition raw score ≤20 (Dunn et al, 2021;Erdodi, 2021;Pearson, 2009); symbol search ACSS ≤ 5 (Erdodi & Abeare, 2020); Trails A T ≤ 33 (Abeare et al, 2019;Ashendorf et al, 2017;Erdodi, 2021); EI-5 = Erdodi Index Five; CR = critical items on the WCT; LR = likelihood ratio (failure rate in the non-White sample divided by the failure rate in the White sample); GMIP = Genuine Memory Impairment Profile.…”

“…This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. (Hurtubise et al, 2020); Coding Age-Corrected Scaled Score (ACSS) ≤5 (Ashendorf et al, 2017;Erdodi & Abeare, 2020); complex ideational material T ≤ 29 ; digit span ACSS ≤ 6 (Erdodi & Abeare, 2020;Hurtubise et al, 2020); letter fluency (FAS) T ≤ 31 ; forced choice recognition trial of the California verbal learning ≤15 ; logical memory recognition raw score ≤20 (Dunn et al, 2021;Erdodi, 2021;Pearson, 2009); symbol search ACSS ≤ 5 (Erdodi & Abeare, 2020); Trails A T ≤ 33 (Abeare et al, 2019;Ashendorf et al, 2017;Erdodi, 2021); EI-5 = Erdodi Index Five; CR = critical items; LR = likelihood ratio; B/P = Borderline over Pass; F/P = Fail over Pass; F/B = Fail over Borderline. The actual combination of critical items is not disclosed here to protect test security.…”

“…is critical item analysis. The assumption underlying critical item analysis is that predictive power is not equally distributed along a scale: certain items contribute more information about the target construct than others (Dunn et al, 2021; Erdodi, Tyson, et al, 2018). Identifying a small subset of items with comparable, better or differential predictive power can abbreviate the testing process, improve classification accuracy or, ideally, both (Denning, 2012).…”