Popular Concussion Assessments' Reliability Using High School and Collegiate Athletes

Background/Purpose: The incidence of sports-related mild traumatic brain injury (MTBI), or concussion, is alarming, with an estimated 1,700,000 MTBIs diagnosed yearly (Faul, Xu, Wald, & Coronado, 2010). Concussion diagnosis is difficult as symptom constellation can vary considerably across athletes. The use of diagnostic tests to determine if a MTBI has occurred and when the individual should return to play (RTP) has been outlined as best practice in concussion consensus statements. Research has shown using two or more tests increases reliability in determining when concussed athletes can RTP (Lee, Sullivan, & Schneiders, 2013). Two commonly used tests are the Immediate Post Concussion Assessment and Cognitive Testing battery (ImPACT™), a computerized diagnostic test that measures neurocognitive functions (memory) and NeuroCom™, which uses force plate technology to measure postural stability (balance). The purpose of this study was to examine the stability reliability of both ImPACT™ and NeuroCom™ in non-concussed high school and collegiate student athletes.

Method: High school (n = 92) and collegiate (n = 73) athletes (male & female; football, basketball, volleyball, soccer, cheer) were tested twice using ImPACT™ and NeuroCom™ approximately 3-4 months between test sessions (pre/post season). Data from subjects who scored an invalid baseline performance, experienced a concussive event within 24 months prior to testing, or reported a diagnosis of ADHD or learning disability were excluded. Composite scores from measurement modules for both ImPACT™ (5 modules) and NeuroCom™ (11 modules) were compared using intraclass correlations to determine reliability. An ideal intraclass correlation measure of 0.9 or higher is expected when determining a clinical diagnostic test (Randolph, McCrea, & Barr, 2005).

Analysis/Results: ImPACT™ composite score intraclass correlations for high school participants (listed first) and college (listed second) were:  Verbal memory=0.65, 0.42; visual memory=0.69, 0.65; visual motor speed=0.79, 0.76; reaction time=0.59, 0.60; and impulse control=0.63, 0.66. NeuroCom™ intraclass correlations for its 11 modules ranged from 0.08 to 0.85 for high school athletes, and 0.13 to 0.67 for college athletes. A few composite scores came close to the 0.9 or higher; however, all results indicated a low to moderate level of reliability.

Conclusions: Basing diagnostic decisions on measures with low to moderate reliability is problematic, and possibly hazardous to the health of concussed athletes. This study found widely utilized ImPACT™ and NeuroCom™ protocols to have scored low to moderate reliability among high school and college student athletes, and calls for further research to revise noted tests and to develop more accurate protocols to identify MTBI.