Evaluating Constructs Represented by Symptom Validity Tests in Forensic Neuropsychological Assessment of Traumatic Brain Injury

Abstract

This study uses a new method to summarize diagnostic validity information to explore which constructs are captured by malingering tests. The Test Validation Summary applies mixed-groups validation to investigate the meaning of test constructs and to estimate test classification characteristics when test validation groups are not "pure" criterion groups (ie, "compliant" vs "malingering"), but members have variable probability of malingering. The method permits the use of tests with relatively low validity to validate tests of greater validity. In our initial analysis, we argue that the Rey 15-Item Memory Test is best construed as an "intention test" (capturing the intention of testtakers when taking a test) as opposed to an "effort test." Using the Test Validation Summary and mixed-groups validation, we demonstrate that as an indicator of "intention to feign cognitive impairment," the Rey 15-Item Memory Test has estimated false-positive rate (FPR) = 0.02 and true-positive rate (TPR) = 0.57. We then explore the meaning of failure on the Word Memory Test (WMT), which uses a dichotomous classification of performance as valid or invalid. Although the WMT is commonly referred to as an "effort test," we argue that it likely captures both "intention" and "effort" but collapses this information into a single dichotomous classification of symptom validity. We demonstrate that, as a result of this dichotomous classification process, the WMT likely has a problematic FPR. In our analysis of previously published WMT data, the WMT FPR is estimated at 0.12 when there is no predisposition to perform poorly but rises dramatically and unrealistically as the predisposition to perform poorly increases. We compare these findings to those of the Validity Indicator Profile (VIP), which captures both intent and effort to classify 4 different sorts of response styles in cognitive testing. In our analyses, the VIP demonstrates that FPR = 0 and TPR = 0.86 when the construct being measured is "intent to perform poorly," and reveals that FPR = 0.06 and TPR = 0.63 when the construct being measured is "inconsistent responding" or "poor effort." We were able to demonstrate for the VIP the same "oversensitivity" shown by the WMT when the VIP was interpreted only as a dichotomous classification test. These results indicate that researchers who attempt to generate classification characteristics for malingering tests must carefully consider what constructs are being captured by the test.

Test effort in persons with Chronic Fatigue Syndrome when assessed using the Validity Indicator Profile

“…Moreover, the VIP classifies performance into one of four categories based on "effort" and "intention," enabling poor performance due to reduced effort (an intention to perform well but with low effort) to be distinguished from poor performance due to deliberate (intention to perform poorly, combined with high effort) or irrelevant (intention to perform poorly, with low effort) responding. The VIP has also been shown to have better false-positive and true-positive rates than other well known effort tests (e.g., Rey 15-Item Memory Test, Word Memory Test; Frederick & Bowden, 2009).…”

“…However, these tests only provide a dichotomous categorization of performance as valid or invalid, with invalid performance often assumed to be intentional. That is, they are unable to differentiate between poor performance due to deliberate intention and reduced effort that occurs for more benign reasons (e.g., fatigue, disinterest; Frederick & Bowden, 2009). Moreover, they do not formally consider the extent to which a person's underlying cognitive capacity impacts on his or her test performance (Frederick, 2000).…”

Inter-Professional Clinical Practice Guideline for Vocational Evaluation Following Traumatic Brain Injury: A Systematic and Evidence-Based Approach

“…grip strength) and observations of inconsistencies within and across test performance [92]. Some examples of symptom validation tests reported in the TBI literature include: The Test of Memory Malingering [116], California Verbal Learning Test [116], The Recognition Memory Test [122], Word Memory Test [118,119] and the Rey 15-Item Memory Test [119].…”

“…automobile insurance, personal injury litigation, workers' compensation) in which vocational evaluation may take place and how this can potentially influence the completion, validity and reporting of evaluation findings. This is particularly relevant in situations where there are financial incentives, secondary gain or in legal, adversarial or forensic practice contexts [115][116][117][118][119][120][121]. The American Academy of Clinical Neuropsychology indicates that neuropsychological assessment should incorporate some formal evaluation of test taking motivation and effort, which may include formal symptoms validity tests, embedded measures within the neuropsychological examination itself (e.g.…”

Functional Neuroimaging of Symptom Validity Testing in Traumatic Brain Injury

“…Contemporary SVT assessment is typically performed with commercially published tests. An underlying assumption in all SVT methods is that the task is 'easy,' even for the neurologically compromised subject (Frederick and Bowden 2009). Valid assessment is operationally defined when SVT performance is at or above some established cut-score, typically recommended by the test's author(s).…”