Utility of VOMS, SCAT3, and ImPACT Baseline Evaluations for Acute Concussion Identification in Collegiate Athletes: Findings From the NCAA-DoD Concussion Assessment, Research and Education (CARE) Consortium

Abstract

Background: The Vestibular/Ocular-Motor Screening (VOMS) is a valuable component of acute (<72 hours) sports-related concussion (SRC) assessments and is increasingly used with the Immediate Post-concussion Assessment and Cognitive Testing (ImPACT) instrument and the third edition of the Sport Concussion Assessment Tool (SCAT3). Research has suggested that VOMS acute postinjury scores are useful in identifying acute concussion. However, the utility of preseason baseline measurements to improve diagnostic accuracy remains ambiguous. To this end, there is a need to determine how reliable VOMS baseline assessments are across years and whether incorporating individuals’ baseline performance improves diagnostic yield for acute concussions. Purpose: To analyze VOMS, SCAT3, and ImPACT to evaluate the test-retest reliability of consecutive-year preseason baseline assessments to directly compare the diagnostic utility of these tools when incorporating baseline assessments versus using postinjury data alone to identify acute SRC. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Preseason and postinjury VOMS, SCAT3, ImPACT Post-Concussion Symptom Scale (PCSS), and ImPACT composite scores were analyzed for 3958 preseason (47.7% female) and 496 acute (≤48 hours) SRC (37.5% female) collegiate athlete evaluations in the National Collegiate Athletic Association—Department of Defense Concussion Assessment Research and Education Consortium. Descriptive statistics, Kolmogorov-Smirnov significance, and Cohen d effect size were calculated. Consecutive-year baseline reliability was evaluated for a subset of 447 athlete encounters using Pearson r, Cohen κ, Cohen d, and 2-way mixed intraclass correlation coefficients (ICCs). Wilcoxon signed rank tests were used to determine the statistical significance between population performances, and the 90% reliable change index (RCI) was calculated from the test-retest results. Preseason to postinjury change scores were then calculated from each tool’s RCI. Finally, receiver operating characteristic (ROC) curve analyses were conducted, and DeLong method was used to compare the area under the curve (AUC) of raw postinjury scores versus change scores from preseason baseline assessments. Potential effects of sex, medical history (learning disorders or attention-deficit/hyperactivity disorder), and outlier data were also explored. Results: Effect sizes were large, and overall predictive utilities were clinically useful for postinjury VOMS Total ( d = 2.44; AUC = 0.85), the SCAT3 Symptom Evaluation total severity score ( d = 1.74; AUC = 0.82), and the ImPACT PCSS total severity score ( d = 1.67; AUC = 0.80). Comparatively, effect sizes were small and predictive utilities were poor for Standardized Assessment of Concussion (SAC), modified Balance Error Scoring System (mBESS), and all ImPACT composites ( d = 0.11-0.46; AUC = 0.48-0.59). Preseason baseline test-retest reliability was poor to moderate ( r = 0.23-0.52; κ = 0.32-0.36; ICC = 0.36-0.68) for all assessments except ImPACT Visual Motion Sensitivity ( r = 0.73; ICC = 0.85). Incorporating baseline scores for VOMS Total, SCAT3 (Symptom Evaluation, SAC, mBESS), ImPACT PCSS, or ImPACT composites did not significantly improve AUCs. Conclusion: VOMS Total and symptom severity (SCAT3, PCSS) total scores had large effect sizes and clinically useful AUCs for identifying acute concussion. However, all tools demonstrated high within-patient test-retest variability, resulting in poor reliability. The findings in this sample of collegiate athletes suggest that incorporating baseline assessments does not significantly increase diagnostic yield for acute concussion.

Acute evaluation of sport-related concussion and implications for the Sport Concussion Assessment Tool (SCAT6) for adults, adolescents and children: a systematic review

“…Value of baseline testing —Seven papers examined the value of baseline testing using the SCAT. In four studies, neither the SCAT2, SCAT3, nor SCAT5 showed an increased value of baseline testing over standard normative and clinical evaluation 52 56 59 60. One paper did show that for the 5-item word list delayed recall portion of the SCAT2, control athletes improved their performance significantly over baseline (consistent with a practice effect), whereas the concussed group did not show an improvement 20.…”

“…The data show that while the overall SMD was lower when collapsed across time points, the greatest differences can be seen within the first 48-hours post-injury. Twenty studies using baseline/post-injury paradigms and including matched controls examined the ability of various iterations of the SAC to assist with the diagnosis of SRC 42–61. Large effects were found when the SAC was administered immediately following concussion 42.…”

“…The VOMS was evaluated in 22 studies, including four high quality studies. The VOMS was usually administered as per Mucha et al ,140 and was found to (1) discriminate concussion from no concussion with high levels of sensitivity, particularly in the acute phase of injury (ie, <24 hours), and (2) identified increased total VOMS symptom scores and/or greater proportion of the sample with scores above the cut-point post concussion52 141–143 even in the absence of baseline testing 52. Seven studies evaluated the function of the VOR with variable results and used several different outcomes.…”

The Post-Concussion Symptom Scale: Normative Data for Adolescent Student-Athletes Stratified by Gender and Preexisting Conditions

“…The normative data provided herein, with stratifications by gender and preexisting health conditions related to baseline symptom reporting, 9,26 provide a reference point for clinicians to help determine what symptom experience may be typical for a student of the same gender with the same health history. This approach may be more accessible than preseason baseline evaluations to assess symptom change, 3 in that baseline symptom evaluations have poor incremental validity beyond postinjury assessments, 19 symptom scores have poor test-retest reliability, 44 and there is a substantial financial and time burden associated with baseline assessments of many student-athletes who will never experience an injury. Prior researchers have recommended symptom checklists, such as the PCSS, in place of baseline assessments for guiding return to play decision making.…”

Repetitive bout of controlled soccer heading does not alter heart rate variability metrics: A preliminary investigation

“…Furthermore, we also had no way of blinding participants to their exposure (heading, sham, or control), which could induce alterations to sympathetic and parasympathetic tone, thereby changing absolute heart rate and potentially HRV metrics. The SCAT3 has recently been shown to have only moderate reliability and specificity in diagnosing concussion and post-concussive symptoms (64) and, thus, its utility in the context of this study can be questioned. Moreover, as the SCAT3 relies on subjective self-report, participants may have been more likely to report symptoms knowing they had just been exposed to a controlled bout of repeated head impacts.…”