Prevalence of Invalid Performance on Baseline Testing for Sport-Related Concussion by Age and Validity Indicator

Abstract

IMPORTANCE Estimated base rates of invalid performance on baseline testing (base rates of failure) for the management of sport-related concussion range from 6.1% to 40.0%, depending on the validity indicator used. The instability of this key measure represents a challenge in the clinical interpretation of test results that could undermine the utility of baseline testing. OBJECTIVES To determine the prevalence of invalid performance on baseline testing and to assess whether the prevalence varies as a function of age and validity indicator. DESIGN, SETTING, AND PARTICIPANTS This retrospective, cross-sectional study included data collected between January 1, 2012, and December 31, 2016, from a clinical referral center in the Midwestern United States. Participants included 7897 consecutively tested, equivalently proportioned male and female athletes aged 10 to 21 years, who completed baseline neurocognitive testing for the purpose of concussion management. INTERVENTIONS Baseline assessment was conducted with the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), a computerized neurocognitive test designed for assessment of concussion. MAIN OUTCOMES AND MEASURES Base rates of failure on published ImPACT validity indicators were compared within and across age groups. Hypotheses were developed after data collection but prior to analyses. RESULTS Of the 7897 study participants, 4086 (51.7%) were male, mean (SD) age was 14.71 (1.78) years, 7820 (99.0%) were primarily English speaking, and the mean (SD) educational level was 8.79 (1.68) years. The base rate of failure ranged from 6.4% to 47.6% across individual indicators. Most of the sample (55.7%) failed at least 1 of 4 validity indicators. The base rate of failure varied considerably across age groups (117 of 140 [83.6%] for those aged 10 years to 14 of 48 [29.2%] for those aged 21 years), representing a risk ratio of 2.86 (95% CI, 2.60-3.16; P < .001). CONCLUSIONS AND RELEVANCE The results for base rate of failure were surprisingly high overall and varied widely depending on the specific validity indicator and the age of the examinee. The strong age association, with 3 of 4 participants aged 10 to 12 years failing validity indicators, suggests that the clinical interpretation and utility of baseline testing in this age group is questionable. These findings underscore the need for close scrutiny of performance validity indicators on baseline testing across age groups.

Influence of Test Environment, Age, Sex, and Sport on Baseline Computerized Neurocognitive Test Performance

“…These results support previous findings from Vaughan and colleagues, 27 which suggested that any previously reported differences in baseline testing performance are more likely a result of test supervision and administration than environment, per se. However, the current finding is in sharp contrast to a recent report from Abeare and colleagues, 1 who reported high failure rates, averaging 56%, for baseline testing using the same tool as the current study. Their finding may have been influenced by their use of large group (ie, average of 20 athletes) test administration, which is contrary to the test manufacturer's recommendations.…”

“…The current finding is also in contrast to the results of Abeare and colleagues. 1 These researchers reported high test failure rates across all age groups, ranging from 29% to 84%, but reported particularly high rates (ie, 84%) for younger age groups (ie, 10-12 years). Differences in the definitions of ''invalid,'' adherence to test manufacturer's administration guidelines and other factors may have influenced the discrepant findings.…”

Sport concussion and attention deficit hyperactivity disorder in student athletes

“…30 In contrast, the literature is not clear as to the effect of ADHD on baseline ImPACT composite scores or validity measures. [31][32][33] In our study, postinjury ImPACT composite scores and cognitive efficiency index were not different between groups. More work is needed to further clarify the effect of ADHD on cognition after SRC.…”

Influence of Special Education, ADHD, Autism, and Learning Disorders on ImPACT Validity Scores in High School Athletes

“…Comorbid ADHD/ LD consistently demonstrated the highest rates of invalid baselines across the standard ImPACT, Higgins, and Manderino indicators. The large discrepancies found between the validity indices in this study are consistent with a prior investigation that have found the Higgins indicators to identify substantially more baselines as invalid than the standard ImPACT indicators in naturalistic samples (Abeare et al, 2018). Because the standard ImPACT and Higgins indicators were created using healthy athletes, it may not be particularly surprising that athletes with NDs are more likely to have performance that is flagged as invalid, not necessarily because of performance validity issues, but possibly due to underlying attention and reading requirements of ImPACT (Lovell, 2018;Schatz et al, 2012).…”

“…Based on these findings, the current study aims to investigate the following research questions: (1) Do the standard and novel embedded validity indicators identify higher rates of invalid baseline performance in high school athletes with ND compared to those with no ND history? (2) Based on previous literature suggesting age (Abeare et al, 2018), gender (Schatz et al, 2012), and sport-based differences in invalid baseline performance, does the addition of ND predict invalid performance above and beyond these other demographic variables? In addition to these questions, multivariate base rates of criterion failure on validity indicators will be presented for athletes with neurodevelopmental history based on the current ImPACT embedded indicators.…”