i

Örebro University

School of Medicine

Degree project 15 ECTS

January 2017

Evaluation of the current

diagnostic tools for concussion

in sports

Version 2

Author: Johanna Wallinder

Supervisor: Pashtun Shahim MD PhD

Department of Neurochemistry

Sahlgrenska University Hospital, Mölndal,

Sweden

2

Abstract

Background: Concussion is a common injury in sports. The need for accurate diagnosis and

follow up is critical to avoid too early return to play and minimize the number of repeated

concussions obtained which can result in post concussions symptoms and chronic traumatic

encephalopathy.

Purpose: The objective of this study was to examine the support for 3 diagnostic tools used

for evaluation of sports-related concussion.

Methods: The tools investigated were Sports Associated Concussion Assessment Tool

(SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and Immediate Post-

Concussion Assessment and Cognitive Testing (ImPACT). This was done by searching three

databases, PubMED, PsychInfo and SportDiscus. The search words were the name of the

specific test, validity and reliability.

Results: A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. Of

the studies evaluated in this study half (7/14) supported the reliability and validity of ImPACT

and the other approximate half (6/14) did not support the reliability and validity of the

ImPACT. SCAT 3 only yielded one article that supported the use of SCAT 3, primarily

within the first week of getting a concussion. Of the 3 articles yielded in RPQ search, two

questioned the construct validity and the third article, which contained two studies supported

it.

Conclusion: None of these diagnostic tests have an unquestionable support within current

literature. Further research regarding the validity and reliability is needed to further develop

the use and understanding of these tools.

3

Abbreviations:

mTBI-mild traumatic brain injury

SCAT3- Sport Concussion Assesment Tool 3

RPQ- Rivermead Post-Concussion Symptoms Questionnaire

ImPACT- Immediate Post-Concussion Assessment and Cognitive Testing

SAC- Standardized Assessment of Concussion

PCS- Post-Concussion Syndrome, GPA-grade point average

ADHD-attention deficit hyperactivity disorder

SBU-statens beredning för medicinsk och social utvärdering

4

CONTENT

Background ........................................................................................................................ iv

Material and Methods ......................................................................................................... iv

Diagnostic tests investigated .............................................................................................. iv

ImPact ................................................................................................................................ iv

SCAT 3 .............................................................................................................................. iv

RPQ .................................................................................................................................... v

Method................................................................................................................................. v

Inculsion and Exclusion Criteria ........................................................................................ v

Data extraction .................................................................................................................. vi

Results ................................................................................................................................ vi

Discussion ........................................................................................................................... x

Conclusion ......................................................................................................................... xii

References ........................................................................................................................ xiii

BACKGROUND

Concussion is a common injury in contact sports such as boxing, American football, rugby

and ice hockey. In the USA, the Centers for Disease Control estimate that 1,6-3,8 million

sports-related concussions occur annually in the United States alone1,2. This figure is,

iii

however, probably underestimated since not all who obtain a mild concussion seek medical

care3.

Concussion is a type of mild traumatic brain injury (mTBI) and is defined as a complex

pathophysiological disturbance of brain function induced by mechanical force or trauma2

4. When the brain is subjected to a rapid change of motion, as when receiving a tackle or hit

to the head, it starts a cascade in the brain cells which includes a change in ionic

concentrations in the cell, glutamate release, energy crisis, axonal damage, cytoskeletal

dysfunction and altered transmission. Inflammation and cell death has also been implicated as

part of the pathophysiology of concussion5.

The symptoms of a concussion include fatigue, headaches, visual disturbances, loss of

memory, trouble concentrating/paying attention and emotional disturbances as irritability and

depression.

At present, there are no valid blood biomarkers or imaging techniques for detection and

diagnosis of concussion. The current clinical imaging biomarkers, such as computed

tomography or magnetic resonance imaging may be performed to rule out intracranial

hemorrhage but contribute little to the diagnostic process. The concussion diagnosis is largely

based on clinical symptoms. Behavioral and cognitive tests may be used to aid the diagnosis,

but with limited specificity6.

Recent data suggest that athletes who have repeated concussions, especially unresolved

concussions with early return to play may be at risk of developing both short-term and long-

term neurological symptoms 2. Short-term risks include delayed reaction time which may

increase risk of further injury and second impact syndrome which includes swelling of the

brain and can therefore be lethal. It is, however, very uncommon7. Long-term risks are

those associated with cognitive impairment and long term neurological consequences,

however, more studies concerning this matter is needed.8

Considering the risks associated with concussions there is an urgent need for objective and

reliable diagnostic and prognostic tools. At present, there are many qualitative and semi-

qualitative tools used to aid in concussion diagnosis including Sports Associated Concussion

Assessment Tool (SCAT3), Rivermead Post Concussion Symptoms Questionnaire (RPQ) and

Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT).

iv

The purpose of this paper was to investigate the evidence of reliability and validity of these

three tests.

MATERIAL AND METHODS

DIAGNOSTIC TESTS INVESTIGATED

ImPact

ImPACT is a computerized concussion management tool and consists of 4 parts, verbal

memory, visual memory, visual motor speed and reaction time. There is also a questionnaire

covering self-reported symptoms. This tool is primarily utilized for baseline testing in the

beginning of the season and post-injury testing. ImPACT was developed from the Pittsburg

Steelers model which was a paper and pencil based test used to monitor Football players in

the National Football League 9. The Pittsburg Steelers model was used until 1998. ImPACT

baselines are usually administered during pre-season and follow up test 1-3 days after the

injury 10.

SCAT 3

SCAT is a standardized test for evaluating athletes with concussion. It was first developed in

2005 and has since been edited and updated to SCAT2 and now SCAT3. The idea for

developing a standardized assessment tool was developed during the consensus meetings held

2001 in Vienna for medical professionals organized by International Ice Hockey Federation,

Fédération Internationale de Football Association and the medical comission of the

International Olympic Committee. The latest version, SCAT3, was one of the outcomes from

the consensus meeting in Zurich 20124.

SCAT3 is mainly used as a sideline assessment tool. Baseline values may or may not be

obtained during preseason. It contains 8 major parts, Glasgow coma scale (testing alertness),

Maddocks score (testing short term memory), self-reported symptoms, Cognitive assessment

(using Standardized Assessment of Concussion, SAC.), neck examination, balance testing

v

(using modified Balance Error Scoring System), coordination and delayed recall. The SCAT 3

form is found in appendix no. 1.

RPQ

Rivermead is a symptom questionnaire where symptoms are self-assessed on a scale of 0-4.

The ratings are subjective comparisons of how the patient was feeling prior to the accident or

within the past 24 hours. The symptoms covered by the questionnaire are headache, dizziness,

nausea/vomiting, noise sensitivity, sleeping disturbance, fatigue, irritability, depression,

frustration/patients, memory, concentration, thought processing, vision and restlessness. It

was developed by N.S. King’s research group in 1995 and is designed to assess the presence

of post-concussion syndrome11. Post-concussion syndrome, PCS, is a when symptoms such

as headache, dizziness appear as a consequence of a mTBI. The RPQ questionnaire is found

in appendix no. 2.

METHOD

Databases PubMed, PsychInfo, SportDiscus were searched using the name of the test (i.e.

SCAT 3, Rivermead or ImPACT) together with realibility and validity.

Since this is a literature study there is no need for ethical considerations or statistical analysis.

Inculsion and Exclusion Criteria

1. Published Articles after 1990

2. Reported outcomes include an analysis of reliability and/or validity of SCAT 3, RPQ

or ImPACT

3. Individuals older than 13years of age

4. Data collected either at baseline or postconcussion

5. English language

6. SCAT, RPQ or ImPACT was used to evaluate sports related concussions

7. Articles must be original studies therefore review articles were rejected

The inclusion criteria no. 6 was disregarded for RPQ since the search yielded no results

related to concussions in sports.

vi

Data extraction

The articles were evaluated with help of the first section of “Mall för kvalitetsgranskning av

observationsstudier” (the GRADE scale was not used) from SBU’s website. The form is

found in appendix no. 3.

RESULTS

A total of 18 articles were selected, 1 for SCAT 3, 3 for RPQ and 14 for ImPACT. A table of

all included articles is found in appendix no. 4

Figure 1: Search for SCAT 3

Figure 2: Search for RPQ

vii

Figure 3: Search for ImPACT

viii

ImPACT

The search yielded 14 articles investigating the validity and reliability of the ImPACT. 10

were within subject cohort studies testing baseline re-testing reliability, three were cross

sectional studies which tested the subject only once and one was a cohort study investigating

differences between concussed and non-concussed athletes.

Of the studies evaluated in this study half (7/14) supported the reliability and validity of

ImPACT 12-17,24 and the other approximate half (6/14) did not support the reliability and

validity of the ImPACT 18-22,25. The reliability, that is that the test will score an athlete

with similar results when tested several times, is tested primarily by baseline testing and

retesting. The validity, meaning that the test measures what it is intended to measure, is harder

to conduct studies since there is no gold standard to compare with. There are studies

comparing ImPACT to other previously validated tests24,25, but as previously mentioned

none of the tests are considered gold standard for diagnosing concussion. The study

examining the rates and predictors for invalid baseline didn’t say whether it supported the test

or not but examined what factors can influence baseline results other than the construct of the

test19.

Baseline testing

There is a debate about whether comparison to baseline testing is the best way to evaluate the

scores of ImPACT or if it is better to compare the scores obtained post-injury to normative

data.

Many of the studies have investigated the test-retest reliability12-22, most of them only with

healthy participants. Study participants have both been athletes and non-athletes12,13, of

varying ages, from high school (13-18) and college students’ (18-24) to professional

players14-25. The results of these studies have, however, been inconclusive. Some studies

have shown high test-retest reliability12-17,24 while others have shown poor reliability18-

22,25. The author Schatz who has conducted several studies12,16 showing reliable test-

retesting within-subjects have criticized, primarily one study showing unreliable test-retesting

conducted by Broglio21, primarily because of the study method in investigating several

neurocognitive tests during the same session. That is, however, not the only study showing

varying results when it comes to testing the reliability of baseline testing 18-20, 22,25.

ix

For baseline-testing to be useful there is need for a valid test-result from the athlete taking the

baseline. ImPACT has as part of the data analysis a function for flagging tests as invalid

showing “++” on the scorecard. That is, if an athlete would for example score 30 or more on

Impulse Control composite their effort would be questioned25. Several predictors for

obtaining an invalid baseline have been identified by Nelson et. al. Several predictors were

studied such as lower GPA (grade point average), diagnosis such as ADHD (attention

deficiency hyperactivity disorder) or learning disability and age. The only predictor for

invalid baseline testing with ImPACT was lower age and that was only a modest association

which became non-significant after adjustment19. There have also been other questions

raised such as how primary language22, prior night’s sleep and so forth can affect the results

of not only baseline but also re-testing26, most of these studies did not, however, turn up in

the searches conducted in this study. Jared Bruce et. al22 raises the question of primary

language in his study of ImPACT’s test-retest reliability in professional hockey players. The

results indicate that the reliability may vary depending on language administration.

Follow up

The use of ImPACT as a tool to follow the concussed athlete’s recovery and as part of the

return to play decision is also not entirely supported in the literature. One study conducted by

L. D. Nelson et. al.23 where concussed athletes completed the ImPACT within 24 hours of

injury and 8, 15 and 45 days’ post-injury and were compared with non-concussed, matched

controls showed only significant differences within the first 24 hours’ and day 8 post-injury.

One study conducted by G. Iverson et. al. tested athletes within 21 days of sustaining a

concussion and found that the ImPACT composite socres testing Processing Speed and

Reaction Time did correlate with a traditional and previously validated test, the Symbol Digit

Modalities Test 24.

SCAT 3

The search for articles investigating the reliability and validity yielded only one article,

“Reliability and Validity of the Sport Concussion Assessment Tool -3 (SCAT3) in High

School and Collegiate Athletes by Esther Y. Chin et. al. In the study, concussed athletes were

x

matched with non-concussed controls and were evaluated 24 hours, 8, 15, and 45 days post-

injury (healthy controls being evaluated at similar times. The results indicated that significant

differences between concussed and healthy controls could be observed primarily 24 hours and

8 days post-injury27.

RPQ

The search for articles investigating the reliability and validity yielded three articles

investigating the construct validity and reliability of the RPQ.

The construct validity of the RPQ, that is if the RPQ is constructed so that it measures what it

is intended to, has been questioned by both S. Eyres et. al. and M. Lannsjö et. al. Both studies

suggest that instead of using one composite score the questions should be divided in to two or

more groups and these groups of questions be scored independent of each other since they

don’t measure the same thing28-29.

The reliability has been tested, as far as the results of this study, only by the developer, N.

King. There are two studies presented in one article, one where the RPQ was administered

twice, once by a professional and once self-administered and one study where the RPQ was

administered twice within one week as a 6 months’ post-injury follow up. Both studies were

considered proving the reliability of the RPQ11 it is, however, questioned by S. Eyres et. al

29.

DISCUSSION

ImPACT

ImPACT is one of the most widely used concussion diagnostic/follow-up tools used in the

USA. It is clear, however, that there is no consensus on the reliability and validity of this test.

Studies show varied results both within and between studies.

The advantages of ImPACT are, it is easy to administer on the computer, it comes in different

languages, and with the online versions scores can be accessed at any computer with access to

xi

internet (older desktop versions are, however, bound to the computer). The different

translations of ImPACT are, however, not studied in depth in terms of reliability. Of note is

also that most of the studies conducted have had any other primary language than English as

an exclusion criteria12-16,20,21,25. Further research of the translations of the ImPACT is

therefore warranted.

Another issue with baseline testing is the risk of some athletes intentionally trying to

underperform on their baseline, so called “sandbagging”19 to be allowed to return to play

sooner since their scores are seemingly not affected by the concussion. This search, however,

did not include those articles and are there -fore not further discussed.

It is a factor to keep in mind that this test is developed by a company and is therefore a

product made to be marketed and sold10. That can be both an advantage, they need a good

product to be able to sell it and remain a good reputation. It can also be a disadvantage in that

there is an interest in making the test look good and perhaps overstating its utility in order to

market it and sell more.

SCAT 3

Since the search only generated one article it is hard to make any conclusive statements.

SCAT 3 has many strengths such as being easy to administer on the side line by filling out the

form with paper and pencil. It doesn’t require a computer or tablet and it assesses more than

neurocognitive functions, for example consciousness and balance which are also affected by

concussions. It is also developed at a consensus meeting by medical professionals and is

available free on the internet, therefore there is no need to sell the test. It does, however, also

have weaknesses in not being evaluated for how all these different tests are to be interpreted

together. As far as this study could illustrate only one study has been made on the entire

SCAT 3, not only the individual parts. There is also no visual testing which is a major part of

other concussion diagnostic tests27.

Therefore, further research is needed of the whole SCAT 3.

xii

RPQ

The RPQ is not primarily used to diagnose concussion but is used to evaluate the presence of

post concussive symptoms. The positive aspects is that it is easy to administer and seems to

have a good reliability, whether self-administered or by a professional11. The reliability

and validity is, however questioned by Eyres et. al. 29 due to the low fit to the statistical

Rasch model used. It could also be questioned though because the studies with the greatest

support for the reliability were performed by the developer of the test and therefore it could

constitute a conflict of interest.

RPQ’s validity is however questionable since many post-concussive symptoms are vague and

includes common symptoms such as headache, nausea and fatigue. The studies support that

all these symptoms can’t be lumped together since the symptoms fit together in a more

complex way28,29.

Limitations

One limitation of this study is that only one person reviewed the articles. Further evaluating

the studies with traditional forms for evaluating diagnostic studies was not optimal since there

is no test that is considered gold standard for concussions. This makes it difficult to design

studies and to evaluate when you have nothing to compare the new diagnostic method to. The

“Mall för kvalitetsgranskning av observationsstudier” wasn’t optimal either since many of the

questions were not applicable or unclear, which made it hard to score the studies. Another

more specific weakness of this study is that this study only searched studies assessing the

whole SCAT 3 test. There are more studies1§ testing the individual parts of SCAT 3 which

could be useful, however that doesn’t mean that the way all these tests are interpreted together

as a whole needs to be further investigated. Also, SCAT 3 has been developed from the SCAT

2 which is similar to the SCAT 3

CONCLUSION

None of the tests discussed in this paper could be considered optimal for diagnosis and

prognosis of concussion in athletes. One test should not be the single base of clinical

decisions made. Consideration of the athletes’ background and circumstances during both

baseline testing and follow up should be part of the return to play decision. Concussions are

complex and the ramifications of too early return to play have not yet been fully understood.

xiii

Also, the determination of what is considered too early return to play is not fully determined.

Is it when symptoms have resolved or when the neurocognitive function is back to baseline

values? Further research in this area is needed to fully understand this problem. The

development and research has also focused a lot on development of imaging methods and

blood biomarkers which could also aid in diagnosis. Perhaps the best way for diagnosing is in

a future where imaging techniques and neurocognitive testing can be combined to avoid the

possible devastating ramifications of too many concussions or too early return to play.

ACKNOWLEDGEMENTS

Special thanks to my supervisor Pashtun Shahim for taking the time to help me write this

paper, also to Yelverton Tegner who helped me get in touch with Pashtun.

REFERENCES

1. Langlois JA, Rutland-Brown W, Wald MM. The epidemiology and impact of

traumatic brain injury: a brief overview. J Head Trauma Rehabil. 2006 Sep-

Oct;21(5):375–378.

2. Harmon KG, Drezner JA, Gammons M, Guskiewicz KM, Halstead M, Herring SA, et

al. American Medical Society for Sports Medicine position statement: concussion in

sport. Br J Sports Med. 2013 Jan;47(1):15-26

3. U.S DEPARTMENT OF HELATH AND HUMAN SERVICES, Centers for Disease

Control and Pervention, National Center for Injury Prevention and Control. Traumatic

Brain Injury in the United States Emergency Department Visits, Hospitalizations and

Deaths 2002-2006. Atlanta (GA): Centers for Disease Control and Prevention,

National Center for Injury Prevention and Control; 2010.

4. McCrory P, Meeuwisse WH, Aubry M, Cantu RC, Dvořák J, Echemendia RJ et al.

Consensus Statment on Concussion in Sport: the 4th Internation Concerence on

Concussion in Sport held in Zurich, November 2012. J Athl Train. 2013 Jul-

Aug;48(4):554-575

5. Giza CC, Hovda D. The New Neurometabolic Cascade of Concussion. Neurosurgery.

2014 Oct;75 (04):24-33

6. Svensson M, Bellander B-M, Undén J, Johnsson H. Traumatiska hjärnskador (TBI)

hos vuxna Internet… Utgivningsort: Internetmedicin 2016 Updated 2016-09-19;

cited 2016-11-30 Available from: http://www.internetmedicin.se/page.aspx?id=1178

xiv

7. Bey T, Ostick B. Second impact syndrome. West J Emerg Med. 2009;10(1):6–10)

8. Gaetz M, Weinberg H. Electrophyiological indices of persistent post-concussion

symptoms. Brain Inj. 2000 Sep; 14(9):815-832

9. Vanderbildt Medical School. The ImPACT Program: History and Development

Internet. Nashville TN 2013 Cited 2016-12-02 Available from:

https://medschool.vanderbilt.edu/vscc-research/files/vscc-

research/public_files/Lovell_Nashville_November2013_.pdf

10. ImPACT Applications, Inc. ImPACT Recourses for Parents and Athletes Internet.

Pittsburg PA 2016 Cited 2016-12-02. Availabel from:

https://www.impacttest.com/audience/?parents-3

11. King NS, Crawford s, Wenden FJ, Moss NE, Wade DT. The Rivermead Post

Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced

after head injury and its reliability. JNeurol. 1995 Sep; 242(9):587-592.

12. Schatz P, Ferris CS. One-Month Test—Retest Reliability of the ImPACT Test Battery.

Arch Clin Neruopsychol. 2013 Aug; 28(5):499-504.

13. Nakayama Y, Covassin T, Schatz P, Nogle S, Kovan J. Examination of the Test—

Retest Reliability of a Computerized Neurocognitive Test Battery. Am J Sports Med.

2015 Aug; 43(8):2018-26.

14. Tsushima WT, Siu AM, Pearce AM, Zhang G. Oshiro RS. Two-year Test—Retest

Reliability of ImPACT in High School Athletes. Arch Clin Neruospychol. 2016

Feb:31(1):105-111.

15. Brett BL, Solomon GS. The Influenve of validity criteria on Immediate Post-

Concussion Assessment and Covgnitive Testing (ImPACT) test—retest reliability

among high school athletes. J Clin Exp Neuropsychol. 2016 Sep 2:1-10

16. Elbin R.J, Schatz P, Covassin T. One-Year Test-Retest Reliability of the Online

Version of ImPACT in High School Athletes. Am J Sports Med. 2011 Nov; 39

(11):2319-2324.

17. Schatz P. Long-Term Test-Retest Reliability of Baseline Cognitive Assessments

Using ImPACT. Am J Sports Med. 2010 Jan; 38(1):47-53.

18. Resch J, Driscoll A, McCaffrey, Brown C, Ferrara MS, Macciocchi S et. al. ImPACT

Test-Retest Reliability: Reliably Unreliable? J Athl Train. 2013 Jul-Aug;48(4):506-

511

xv

19. Nelson LD, LaRoche AA, Pfaller AY, Lerner EB, Hammeke TA, Randolph C et. al.

Prospective, Head-toHead Study of Three Computerized Neurocognitive Assesment

Tools (CNTs): Reliability and Validity for the Assessment of Sport-Related

Concussion. J Int Neuropsychol Soc. 2016 Jan;22(1):24-37

20. Register.Mihalik JK, KontosDK, Guskiewicz KM, Mihalik JP, Conder R, Shields EW.

Age-Related Differences and Reliability on Computerized and Paper-and-Pencil

Neurocognitive Assessment Batteries. J Athl Train. 2012 May-Jun;47(3):297.305

21. Broglio SP, Ferrara MS, Macciocchi SN, Baumgartner TA, Elliot R. Test-Retest

Reliability of Computerized Concussion Assessment Programs. J Athl Train 2007

Oct-Dec;42(4)509-514

22. Bruce J, Echemendia R, Meeuwisse W, Compier P, Sisco A. 1 year test-retest

reliability of ImPACT in professional ice hockey players. Clin Neuropsychol. 2013

Dec;28(1):14-25

23. Nelson LD, Pfaller AY, Rein LE, McCrea MA. Rates and Predictors of Invalid

Baseline Test Performance in High School and Collegiate Athletes for 3

Computerized Neurocognitive Tests:ANAM, Axon Sports and ImPACT. Am J Sports

Med. 2015 Aug;43(8);2018-2026

24. Iverson GK, Lovell MR, Collins MW. Validity of ImPACT for Measuring Processing

Speed Following Sports-Related Concussion. J Clin Exp Neuropsycol. 2005

Aug;27(6):683-698.

25. Allen BJ, Gfeller JD. The Immediate Post-Concussion Assessment and Cognitive

Testing battery and traditional neuropsychological measures: A construct and

concurrent validity study. Brain Inj. 2011;25(2):179-191.

26. McClure DJ, Zuckerman SL, Kutscher SJ, Gregory AJ, Solomon GS. Baseline

neurocognitive testing in sports-related concussion: the importance of a prior night’s

sleep. Am J Sports Med. 2014 Feb; 42(2):472-478.

27. Chin EY, Nelson LD, Barr WB, McCrory P, McCrea MA. Reliability and Validity of

the Sport Concussion Assessment Tool—3 (SCAT3) in High School and Collegiate

Athletes. Am J Sports Med. 2016 Sep;44(9):2276-2285.

28. Lannsjö M, Borg J, Björklund G, Af Geijerstam JL, Lundgren.Nilsson A. J Rehabil

Med. 2011 Nov;43(11):997-1002.

29. Eyres S, Carey A, Gilworth G, Neumann V, Tennant A. Construct validity and

reliability of the Rivermead Post-Concussion Sumptoms Questionnaire. Clin Rehabil.

2005 Dec;19(8)878-887

xvi

Appendix no. 1.

xvii

xviii

xix

xx

Appendix no. 2.

xxi

Appendix no. 3.

xxii

xxiii

ii

Appendix no 4.

Publication

(Author, year,

referencenumber)

Type of

Study

Patient

characteristics

Exposure or

intervention

Compared to Measure of effect Results Quality

assessment

Schatz P, 2010,

17

Case series College

Athletes

Baseline

testing with

ImPACT

approximately

2 years apart

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Baseline values

remain fairly stable

over a 2-year period

Risk for

systemic bias:

Moderately high

Nelson LD,

2015, 23

Prospective

head-to-

head

High School

and collegiate

athletes

Concussed

athletes

completing

ImPACT

Non-concussed,

matched controls

completing

ImPACT

Reliable change

indices (RCI)

Computerized

neurocognitive

testing may

contribute to

identification within

24h of injury but

little after symptom

reliefe

Risk for

systemic bias:

Moderately high

iii

Nelson LD,

2016, 19

Cross-

sectional

High School

and collegiate

athletes

Baseline

testing with

ImPACT

Number of

baselines classified

as invlaid

Validity criteria

may not be

sensitive to detect

the cause of invalid

baseline scores.

Risk for

systemic bias:

Moderately high

Iverson GL,

2005, 24

Cross-

sectional

Amature

athletes

ImPACT

testing post

Injury

Symbol Digit

Modalities Test

(SDMT)

Correlation between

the two test-scores

Speed and reaction

time composite

scores of ImPACT

seem to measure

similar construct as

SDMT

Risk for

systemic bias:

Moderately high

Broglio SP,

2007, 21

Repeated-

measure

University

students

Baseline Test

and Re-testing

of ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Tests showed low

to moderate test-

retest reliability

Risk for

systemic bias:

Low

Bruce J, 2013, 22 Repeated-

measure

Professional

ice hockey

players

Baseline Test

and Re-testing

of ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Mixed support for

visual motor and

reaction time

composite scores,

verbal and visual

composite scores

may not be

sensitive to clinical

change

Risk for

systemic bias:

Low

iv

Register-Mihalik,

2012, 20

Cohort

study

High school

and collegiate

athletes

Test and Re-

testing of

ImPACT

Differences in

composite scores

between age-groups

and between test

sessions

Performance may

vary between

sessions

Risk for

systemic bias:

Low

Resch J, 2013, 18 Cross-

sectional

Men and

women ages

18-24

Test and Re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Variable test-retest

reliability.

Risk for

systemic bias:

Moderately high

Brett BL, 2016,

15

Repeated

measure

High school

athletes

Test and re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Application of strict

validity criteria

does not affect rest-

retest reliability

Risk for

systemic bias:

Moderately High

Tsushima WT,

2016, 14

Repeated

measure

High school

athletes

Test and re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC), Reliable

change index(RCI)

Study indicated that

baseline values

were stable within a

2 year interval

Risk for

systemic bias:

Moderately High

v

and regression

based method(BM)

Nakayama Y,

2014, 13

Repeated

measure

College

students

Test and re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC),

Study indicated that

ImPACT has good

reliability

Risk for

systemic bias:

Moderately High

Allen BJ, 2011 Cross-

sectional

College

students

ImPACT

testing

NFL’s traditional

neurospychological

tests

Factor analysis

examined

component

structure.

Correlations

analyses examines

relationships

between variables

within and in-

between tests.

Study indicated that

there is both

overlapping in their

constructs as well

as notable

differences

Risk for

systemic bias:

Low

vi

Schatz P,

2013,12

Repeated

measure

College

students

Test and re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC) and Pearson’s

correlation

coefficients

Repeated testing

may lead to

significant

improvement on

composites such as

visual motor speed

but does not lead to

practice effect on

memory or reaction

time.

Risk for

systemic bias:

Moderately High

Elbin R.J, 2011,

16

Repeated

measure

High school

athletes

Test and re-

testing of

ImPACT

Differences

between individuals

2 baseline scores

measured with Intra

Class Correlation

(ICC)

Reliability of the

online version of

ImPACT is stable

and better than the 2

year reported from

desktop version-

Risk for

systemic bias:

Low

Eyres, 2004, 29 Cross-

sectional

Adults

visiting the

emergency

room after

head trauma

Completing

RPQ post

injury

RPQ score analysed

with Rasch model.

RPQ is not up to

modern

psychometric

standards.

Risk for

systemic bias:

Moderately high

vii

Lannsjö, 2011,28 Cross

sectional

Patients with

mild

traumatic

brain injury

(mTBI)

visiting the

emergency

room, ages 6

years and

older

Completing

RPQ post-

injury

RPQ score analysed

with Rasch model

RPQ is not optimal

for patients with

mTBI. It is,

however,

unsensitive to age

and gender.

Risk of systemic

bias: Low

King NS, 1995,

11

Repeated

measure

Head injury

patients

Completing

RPQ twice

post-injury

Comparing the two

RPQ scores with

Spearman rank

correlation

RPQ shows good

reliability

Risk for

systemic bias:

Moderately high

Chin EY, 2016 Cohort

diagnostic

study

High school

and collegiat

athletes

Concussed

athletes

completing

SCAT 3 at

baseline and 4

times post

injury

Non-injured

matched controls at

same time as

injured athletes.

Comparing injured

with non-injured

matched controls

with RCI

When interpreting

the SCAT3 results,

individual

differences should

be considered

Risk for

systemic bias:

Moderately high