Factors Affecting Athletes' Performance on Baseline and Post Concussion Cognitive Testing Philip Schatz, PhD Saint Joseph’s University Tracey Covassin,

Factors Affecting Athletes'

Performance on Baseline and

Post Concussion Cognitive

TestingPhilip Schatz, PhD

Saint Joseph’s University

Tracey Covassin, PhD, ATCMichigan State University

Anthony Kontos PhDHumboldt State University

Elizabeth Larson BSHumboldt State University

RJ Elbin MAMichigan State University

National Academy Of Neuropsychology - November 2009

Overview

• Methodological Issues – Philip Schatz• History of Concussion – RJ Elbin• Learning Disabilities – Beth Larson• Depression and Anxiety – Anthony Kontos• Gender and Age – Tracey Covassin

Methodological Issues(keeping the “psychology” in sports concussion assessment)

-Scheduling of baselines

-Individual vs. group effects

-Environmental distraction

-Feedback from test takers

Concussion Testing

(It all started with) Barth’s VA football study:

• within-subjects comparison • changes from baseline performance

• between-groups comparison • matched cohorts

• Model widely adopted (colleges, NFL, NHL)

Computer-based Assessment

Computer-based tests introduced (circa 2000)ImPACT, CogSport, CRI, ANAM

Advantages:• ease of administration• automated scoring• decreased practice effects• increased test-retest reliability

Collie,et al. (2003). Br J Sports Med, 37, 556-559

Baseline Testing

Baseline pre-injury testing, serial follow-up is a necessary component

CISG: Vienna (2001), Prague (2004), Zurich (2009)

AAN: (1997: Neurology, 48, 581-585): …recommend neuropsych measures to detect impairment associated w/concussion

NATA: (2004: Guskewicz et al., Neurosurg, 55, 891-895)

neuropsych data assists ATCs in evaluating recovery

Scheduling of Baselines

Question: How often to update baselines?

After a concussion, start of academic yearCollie: (2001: BJSM, 35, 297-302)Valovich: (2003: JAT, 38, 51-56)

More frequent for “youth” athletes-compared to “older” athletes-period of rapid cognitive maturation (8-15)(Buzzini, 2006: Curr Opin Pediatr; Maruff 2004: BJSM; McCrory: 2004: BJSM)


How often to update?

Annually, independent of age, until long-term test-retest reliability data are available

Randolph, McCrea, Barr: (2005: JAT, 40, 139-152)

Test-Retest Data: (ImPACT)7-day (Iverson, Lovell, Collins: 2003: Clin Neuropsy)

45/50-day (Broglio, et al.: 2001: JAT)

4-month (Miller, et al.: 2007: AJSM)


Schatz: 2009: AJSM

2-year test-retest reliability of ImPACT

117 Varsity Athletes, base-lined 2 years apart-22 concussed between baselines-7 excluded due to high Impulse Control

Resultant sample of 95 (no football players)


Schatz: 2009: AJSM

Mean age Time 1: 18.8, Time 2: 20.8

54% male, 46% female

Mean difference of 1.9 years between baselines


Comparing Time 1 -> Time 2-Pearson’s r, t-test-Intraclass Correlation CoefficientICC can distinguish those sets of scores that are merely ranked in the same order from test toretest from those that are not only ranked in the same order but are in low, moderate, orcomplete agreement; also yields UER (consistency across baselines; measurement instrument as source of error?)

-Reliable Change Indices (RCI)Is change between repeated assessments reliable and meaningful; an estimate of the probability that a score was not obtained as a result of measurement error

-Regression-based MeasuresScores from the first assessment are placed into a regression analysis, using the score at Time 2 as the DV; the resulting equation provides an adjustment for the effect of initial performance level, as well as controlling for any regression to the mean


Time 1 -> Time 2-Pearson’s r – little help (.27-.60)-t-test – no significant differences-Intraclass Correlation Coefficient(ICC as well as UER)

Processing Speed (.75); Reaction Time (.68), Visual Memory (.66), Verbal Memory (.47), Symptom scores (.44).


T1 -> T2: % athletes showing signif. Change-Reliable Change Indices (RCI)(“Traditional” /“Chelune” method – adjust for practice effects)

80% CI 95% CIVerbal Memory 8% / 12% 5% / 5%Visual Memory 13% / 10% 9% / 7%Process Speed 7% / 10% 4% / 2%Reaction Time 7% / 9% 5% / 5%Symptoms 8% / 8% 6% / 6%

Nearly identical numbers of athletes showed improvement &declines in performance from Time 1->Time 2


T1 -> T2: % athletes within expect. limits-Regression-based Measures (RMB)

80% CI 95% CIVerbal Memory 94.7% 97.9%Visual Memory 94.7% 97.9%Process Speed 94.7% 96.8%Reaction Time 96.8% 97.9%Symptoms 89.5% 94.7%

Nearly identical numbers of athletes showed improvement &declines in performance from Time 1->Time 2

Scheduling of Baselines: Summary

• 2-year test-retest data are stronger than published data at 7-50 days• 2-year test-retest data show stability• Using RCI, only a small % of athletes showsignificant change – Verb Mem, Symptoms• Using RBM, no athletes showed signif change

• Stretching time between baselines from 1 to 2years may have little effect on concussionmanagement in collegiate athletes

Individual vs. group effects

Benefits of computer-based testing:• cost-benefit gains• increased security• savings in time, money• rapid testing of an entire team

(French & Beaumont: 1987: Br. J Clin Psy; Barak: 1999: Appl Prev Psych: Collie: 2001 BJSM; Collie &Maruff: 2003: BJSM)


Role of distraction in group setting?(Ligon: 1942: Educ and Psychol Measurement)

• Common errors in group tests:• misunderstood instructions, carelessness,• low motivation, dishonesty of subjects• mental confusion due to too great excitement, • size of group • group inter-distraction.

To reduce errors, “training of group testers is needed


Role of distraction in group setting?• SATs, GREs, licensing exams=precedent?• Ext. distraction may affect performance (McCrory 2005)

• Test should be administered in conditions without noises or disruptions (Pierce & Pierce: 1963: J Consult Psychol)

• Wechsler (1955): Conducted in settings that are “quiet and disciplined”• Studies show no difference between individual and group admin (PPVT: Morris: 1960:J Educ Psych; WAIS: Eme & Walker: 1969, J Clin Psych; Bender-Gestalt:Brannigan:1995: Percep Mot Skills; WAIS/IPET: Neva & Salo: 2003)


Role of distraction in group setting?

Schatz, Neidzwski, Moser, Karpf (in revision):

Evaluated subjective feedback from athletes taking ImPACT• Environmental: noise (talking, laughing, cell phones ringing), distractions (keyboards clicking), discomfort (room temperature)

• Also recorded Computer-based and Instruction-based (problems) feedback

• 538 out of 1818 HS student provided subjective feedback (32.4%)

• 161 reported environmental distraction (9.7%)




Evaluated subjective feedback from athletes taking ImPACT• Students reporting environmental feedback endorsed signif. moreSymptoms (8.8 vs. 5.9; p<.001; d=.21)

• physical symptoms (headache, fatigue, difficulty falling asleep, drowsiness, sleeping less), • cognitive symptoms (feeling slowed down, mentally foggy, difficulty concentrating, difficulty remembering)• emotional symptoms (feeling irritable, sadness, nervousness, feeling more emotional).


What to do about distraction in group setting?

• test individually • seat athletes every-other computer (Broglio: 2007)

• make sure someone in room has “authority” • Team captain, Coach, Head ATC

• explain the process, educate athletes• recourse for inappropriate behavior

•Team is re-tested, not cleared to play until testing is complete



Need counterbalanced basic methodological design

• Group->Group• Group->Individual• Individual->Group• Individual->Individual

Subjective Feedback

Benefits of computer-based testing:

“the freedom to increasingly focus on treatment or qualitative assessment gained by the automation of data collection…”

American Psychological Association. (1986). Guidelines for Computer-based Tests and Interpretations. Washington, DC: APA

Subjective Feedback

If athletes are tested in groups, where is the “testing of the limits” during baseline exams?

• Are (neuro)psychologists even present?• Do (neuro)psychologists need to be present?• What qualitative data can be obtained?

Subjective Feedback

Who completes baseline assessments?

• team physician or web-based (CISG/Vienna, 2002)

• Neuropsychologists in best position to interprettest data/batteries (CISG/Zurich, 2008)

• RTP decision is medical – results assist RTPdecisions (CISG/Zurich, 2008)

• Non-neuropsychologist personnel can administertests (Moser, et al/NAN PoP, 2007)

• Interpretation of results restricted to neuropsych-ologists (Moser, et al/NAN PoP, 2007)

Subjective Feedback

What qualitative data can be obtained?

• Subjective feedback from test takers (ImPACT)

• Environmental• Computer• Instructions

• Completed at the end of the examination

Subjective Feedback

Role of subjective feedback


Evaluated subjective feedback from athletes taking ImPACT• Computer-based feedback: mechanical problems with the mouse, other issues (problems with the monitor, pop-up messages).

• Problems with test instructions: difficulty understanding specific modules of the test, general issues (confusion, failure to read instructions, not devoting enough time).

Subjective Feedback

Role of subjective feedback?


• 538 out of 1818 HS student provided subjective feedback (32.4%)

• 199 reported computer problems (12%)• 297 reported problems with instructions (17.9%)

Subjective Feedback

Students reporting computer problems had signif. Faster reaction time (53 vs. 54; p<.004; d=.14) (statistically but perhaps not clinically significant)

Baddaley and colleagues (1993): a form of attentional filtering –allows subjects to concentrate on relevant items and exclude irrelevant stimuli.

Relatively infrequent, arousing, and isolated problems may actually serve to increase arousal levels, thus optimizing performance (e.g., Yerkes-Dodson, 1908).

Subjective Feedback

Students reporting problems with instructions endorsed signif. more symptoms (7.8 vs. 5.9; p<.004; d=.19)

• physical symptoms (balance problems, fatigue, drowsiness, sleeping less), • cognitive symptoms (feeling slowed down, difficulty concentrating)• emotional symptoms (feeling irritable, feeling more emotional).

Related to history of concussion:None: 16.7% Previous: 23.6%[X2(1)=7.9; p=.005]

Subjective Feedback

• Increased symptoms may reflect test-taking anxiety, exacerbation of existing “issues” from testing

• Incorporating a question and answer session following testing may enhance qualitative data.

• Findings reflect the importance of understanding concussion-based symptoms and their relationship to baseline symptoms and an athlete’s personal style

FACTORS THAT AFFECT NEUROCOGNITIVE TESTING:

HISTORY OF CONCUSSION

Overview of Research: History of Concussion(s)

• Numerous studies have investigated the influence of concussion history on the risk and recovery from future concussion– Three questions have emerged…

• Are athletes with a Hx of concussion at a greater risk for future concussion?

• Do athletes with a Hx of concussion take longer to recovery from subsequent concussion?

• Are their any long-term effects associated with a Hx of multiple concussions?

Are Athletes With a History of Concussion at a Greater Risk for Future Concussion?

• Athletes with a history of concussion are at a four to six times greater risk for incident concussion (Zemper, 2003; Wilberger, 1993; Gerberich et al. 1983)

• Concussed H.S. and collegiate athletes are 3x times more likely to sustain another concussion in the same season (Guskiewicz et al. 2000)

• Dose Response for # of previous concussions and risk for incident concussion (Guskiewicz et al. 2003)

– 3+ (3.4x)– 2 (2.8x)– 1 (1.5x)

• Hx of 3+ and severity of future concussion (Collins et al. 2002)

– 9.3 times more likely to demonstrate 3-4 abnormal markers– On-field LOC (6.7x), confusion (4.1x), anterograde amnesia (3.8x)

Do Athletes with a History of Concussion Take Longer to Recover from Subsequent Concussion?

• No NC differences between 1st and 2nd concussion (Macciocchi et al. 2001)

• Prolonged recovery after two concussions?– 2+ Hx of concussion

• Lingering memory impairment and slower reaction time up to 5 days post-concussion (Covassin et al. 2008)

– 3+ Hx of concussion• Significant memory impairment at 2 days post-concussion

(Iverson et al. 2004)

• Prolonged symptom resolution associated with 3+ Hx (Guskiewicz et al. 2003)

Are There Long-Term Effects Associated with a History of Multiple Concussions?

• Assessing NC function in athletes with prior history of concussion – Deficits in executive function and processing speed

observed in Hx of 2+ compared to 1 or zero (Collins et al. 1999)

– No differences on NC performance between H.S. athletes w/Hx 2+ and recently concussed (w/in 2 weeks) (Moser et al. 2005)

• Support found in studies using paper-and-pencil NC tests

Are There Long-Term Effects Associated with a History of Multiple Concussions?

• No support for computerized NC tests– Studies using ImPACT and CRI found no differences

between groups of athletes with and without hx of concussions (0, 1, 2, 3, 4) (Collie et al. 2006; Iverson et al. 2006)

• Are paper-and-pencil better at detecting long-term impairment than computerized tests?– No differences between Hx vs. No Hx on computerized and

formal NC tests (Bruce & Echemendia, 2009)

• If long-term effects do exist, then NC testing may not be sensitive enough to detect residual impairment (Broglio et al. 2006)

• Future study– Neuro-imaging

Linking Research to Practice

• Current literature suggests that athletes with a Hx of concussion are at a greater risk for injury and may take longer to recover from subsequent concussion

• Obtaining detailed information on previous concussion history is important!– Identify athletes who are at a higher risk for

incident concussion– Educate them about concussion

What is the best practice to obtain this information?

• Do we take the athletes word for it?– Limitations of self-report/recall can be misleading– Concussions are under-reported at all athletic levels (Kaut et

al. 2003; LaBotz et al. 2005)

• Pre-participation screening (PPS) forms?– Lacks specific questions regarding concussion hx

(McCrory, 2004)

– Terminology issue (e.g., “ding” and/or “bell-ringer”)• 8.5 % said yes when asked if they have had a

“concussion”• 25% said yes when using lay terminology (Valovich-

McLeod, 2008)

What is the best practice to obtain this information?

• Concussion Symptom Scale (CSS)– Higher potential for estimating concussion Hx than PPS

(Valovich-McLeod, 2008; LaBotz et al. 2005)

– The endorsement of one Sx does not imply concussion• May over-report concussion Hx

– Interpretation is key!– Headache and blacking out strongly associated with

concussion Hx on PPS• Other sources and questions?

– Coaches and teammates are unreliable (McCrory, 2004)

Clinician Recommendations:Detailed Concussion History

• Include…– Specific questions about previous S/S of concussion

NOT just the perceived number– Previous head, face, neck injuries

• Conconmitant concussion?

– Symptom severity vs frequency of impacts• Indicate vulnerability?

– Consider “lay” terminology • E.g., bell-ringer

– Use concussion symptom scale forms• Inquire for both sport and non-sport injuries

Gender and Age Differences

Sport-related Concussion Research: Exploring Gender

• The majority of concussion research in the past has been conducted on males

• Female sport participation has increased at both the collegiate and high school level

• Are there gender differences in:– Risk– Concussion symptoms– Post-concussion outcomes

Gender Differences and Risk of Concussion in Collegiate Athletes

• Females have a greater incidence of concussion than males

(Covassin et al. 2003; Gessel et al. 2007;Hootman et al. 2007; Colvin et al. 2009)

• Concussions represent 5 to 21% of all reported game injuries for NCAA female athletes and 2 to 9% of all reported game injuries for NCAA male athletes (Hootman et al. 2007)

Women’s ice hockey 21.6%Men’s ice hockey 9.0%

Women’s lacrosse 9.8%Men’s lacrosse 8.6%

Women’s soccer 8.6%Men’s Soccer 5.8

Football 6.8%

Gender Differences and Risk of Concussion in High School Athletes

• Rechel et. al. (2008) reported a greater proportion of competition injuries 12.0% compared to practice injuries 5.9%

Girls basketball 19.0%

Girl’s soccer 18.8%

Boy’s soccer 15.6%

Gender Differences and Concussion Symptoms

• Female concussed athletes reported more concussion symptoms than male concussed athletes (Broshek et al. 2005)

– Poor concentration– Increased fatigue– Lightheadedness– “Seeing stars”

Gender Differences and Concussion Symptoms

• Recent study on high school and collegiate soccer players found concussed female athletes demonstrated more symptoms than concussed male athletes (Colvin et al. 2009)

– Total concussion symptoms– Headache

Gender Differences and Concussion Outcome

• Female concussed athletes have demonstrated longer recovery patterns than concussed male athletes– visual memory (Covassin et al. 2007)

– slower reaction time (Colvin et al. 2009; Broshek et al. 2005)

What may Account these Gender Differences?

• Females have decreased head mass and neck strength compared to males (Tierney et al.

2005;Manshell et al. 2005; Broglio et al. 2003)

• Females may report their concussion symptoms with a greater frequency than males

Age Difference and Concussion

• High school athletes have been found to demonstrate a slower neurocognitive and symptom recovery following concussion than collegiate athletes (Field et al., 2003; Lovell et al., 2003; McCrea et al., 2003; Sim et al. 2008)

Age Differences and Recovery from Concussion

• High school athletes exhibited memory impairment up to 14 days post-concussion (Field et al. 2003; McClincy et al. 2006)

• Collegiate athletes demonstrate memory impairment up to 5 days post-concussion (Macciocchi et al. 1996;Field et al. 2003)

Age Differences and Concussion

• Immature or developing brain places high school athletes at risk for more adverse outcomes following concussion

• Larger head-to-body ratio• Immature musculoskeletal systems (i.e.,

weaker and smaller)• Larger subarachnoid space in the cranium

(e.g., allowing more room for brain movement)

Clinical Implications

• Concussed high school athletes should be removed from practice or competition (Guskiewicz

et al. 2004)

– Second impact syndrome

Documents

Factors Affecting Athletes' Performance on Baseline and Post Concussion Cognitive Testing Philip Schatz, PhD Saint Joseph’s University Tracey Covassin,