Minimizing evaluation time while maintaining accuracy: Cross-validation of the Test of Memory Malingering (TOMM) Trial 1 and first 10-item errors as briefer performance validity tests.

Abstract

Current standards of practice in neuropsychology advocate for including validity tests (PVTs). Abbreviating PVTs, such as the Test of Memory Malingering (TOMM), may help reduce overall evaluation time while maintaining diagnostic accuracy. TOMM Trial 1 performance (T1), as well as the number of errors within the first 10 items of Trial 1 (TOMMe10), have shown initial promise as abbreviated PVTs but require additional external cross-validation. This study sought to replicate findings from other mixed, diverse, clinical samples and provide further validation of abbreviated administrations of the TOMM. Data included 120 veterans who completed the TOMM and 3 criterion PVTs during clinical evaluation. In total, performance from 68% of the sample was classified as valid (52% met criteria for cognitive impairment), and performance from 32% of the sample was invalid. Group differences, diagnostic accuracy statistics, and receiver operating characteristic (ROC) curves were analyzed for relevant TOMM indices. There were large (ηp2 = .45–.66), significant differences between validity groups (p < .001) on TOMM T1 and TOMMe10, with lower TOMM T1 and higher TOMMe10 scores for participants with invalid performance. Using established cut-scores, sensitivities/specificities were: TOMMe10 ≥1 error: .84/.66; ≥2 errors: .74/.93; TOMM T1 ≤40: .82/.93. ROC curve analysis yielded significant areas under the curve for both TOMMe10 and T1 with respective optimal cut-scores of ≥2 errors (.74 sensitivity/.93 specificity) and ≤41 (.84 sensitivity/.91 specificity). TOMMe10 and T1 performances are minimally impacted by cognitive impairment. Although both evidenced robust psychometric properties, TOMM T1 continued to show greater accuracy than TOMMe10.

Comparing the Independent and Aggregated Accuracy of Trial 1 and the First 10 TOMM Items for Detecting Invalid Neuropsychological Test Performance Across Civilian and Veteran Clinical Samples

“…T1e10: Test of Memory Malingering Trial 1 first 10 items; T1: Test of Memory Malingering Trial 1. See Kraemer et al (2020) for the T1e10 failure cut-score and Martin et al (2020) for the T1 failure cut-score.…”

“…The psychometric strength of using TOMM T1 on its own as a robust measure of performance validity has been well-established across a variety of clinical settings and methodological approaches (Denning, 2012, 2021; Hilsabeck et al, 2011; Kraemer et al, 2020; Martin et al, 2020; Webber et al, 2018). Specifically, T1, when used alone, has demonstrated utility among community volunteers, veterans, civilians, older adults, and criminal forensic referrals.…”

“…The TOMM is one of the most frequently administered and widely researched of all PVTs, with strong empirical support across a wide array of clinical and medicolegal populations (e.g., Martin et al, 2020). Although the standard administration of this test involves two trials, T1 has been shown to have robust utility as an independent PVT with accuracy rates that match the more traditionally used T2 (e.g., Cohen et al, 2021; Denning, 2012; 2021; Fazio et al, 2017; Greve et al, 2006; Martin et al, 2020; Kraemer et al, 2020; Webber et al, 2018). The use of the first ten items of T1 (T1e10) has also received growing research support as an even briefer measure (Denning, 2021; Grabyan et al, 2018; Kulas et al, 2014; Ovsiew et al, 2021).…”

“…However, the full TOMM can take upward of 15–20 minutes to administer, leading to an exploration of abbreviated versions such as performance on TOMM Trial 1 (T1; e.g., O’Bryant et al, 2007) and number of errors on the first 10 items of T1 (T1e10; Denning, 2012). Numerous studies have supported T1’s robust classification rates for identifying invalid performance when used as an independent PVT in clinical and forensic contexts (e.g., Cohen et al, 2021; Denning, 2012; 2021; Fazio et al, 2017; Kraemer et al, 2020; Ovsiew et al, 2021; Webber et al, 2018). In fact, several studies have shown that T1 has equivalent accuracy and associated psychometric properties as the traditional TOMM Trial 2 (T2), even though T1 halves the test administration time burden (Greve et al, 2006; Martin et al, 2020; O’Bryant et al, 2008; Ovsiew et al, 2021; Schroeder et al, 2019).…”

A Known-Groups Validation of the Medical Symptom Validity Test and Analysis of the Genuine Memory Impairment Profile

“…In particular, fulfilling the expectation of administering multiple PVTs across an evaluation may add substantial time to an already time-limited process. Although embedded PVTs offer one solution to this issue by integrating validity indicators into existing neuropsychological tests, increasing the administration efficiency of freestanding measures is also desirable, so long as high sensitivity for detection of invalid performance is maintained (e.g., Denning, 2012; Kraemer et al, 2020). Evaluating the accuracy and psychometric properties of such abbreviated measures is therefore imperative.…”