American Chiropractic Neurology Board, Inc. (ACNB) … · American Chiropractic Neurology Board, Inc. (ACNB) 2015 Diplomate Examination Technical Report April 10, ... Written Examination

American Chiropractic Neurology Board, Inc.

(ACNB) 2015 Diplomate Examination Technical

Report

April 10, 2016

Tina Freilicher, Ph.D., Shoreline Psychometric Services, LLC.

2

Table of Contents

INTRODUCTION 4

SUMMARY STATISTICS FOR THE WRITTEN EXAMINATION 4

ANALYSIS OF SUBCOMPONENT DATA 7

OVERVIEW OF ITEM ANALYSIS STATISTICS 9

SUMMARY STATISTICS FOR THE PERFORMANCE EXAMINATION 11

OVERVIEW OF ITEM ANALYSIS STATISTICS 11

QUALITY CONTROL CHECKS OF THE DATA 13

SUMMARY AND RECOMMENDATIONS 13

REFERENCES 15

Index Of Tables

Table 1 - Written Examination Summary Statistics .................................................................... 5

Table 2 – Written Examination Frequency Distribution of Total Test Scores .......................... 5

Table 3 - Written Examination Performance on Parts I and II .................................................. 6

Table 4 - Written Examination Percent & Number of Items by Work Activities ...................... 7

Table 5 - Written Examination Percent & Number of Items by Content Domain .................... 7

Table 6 - Written Examination - Work Activities Sub-Score Statistics ...................................... 9

Table 7 - Written Examination - Content Domain Sub-Score Statistics .................................... 9

Table 8 - Performance Examination Statistics .......................................................................... 12

Table 9 – Written Examination Work Activity: Diagnosis – Frequency Distribution of Scores ...............................................................................................................................................

31

Table 10 - Written Examination Work Activity: HISTORY - Frequency Distribution of Scores ...............................................................................................................................................

32

Table 11 - Written Examination Work Activity: PHYSICAL EXAM - Frequency Distribution

of Scores ............................................................................................................................... 32

Table 12 - Written Examination Work Activity: SPECIAL STUDIES & REFERRAL -

Frequency Distribution of Scores ....................................................................................... 33

Table 13 - Written Examination Work Activity: TREATMENT/REHABILITATION -

Frequency Distribution of Scores ....................................................................................... 33

Table 14 - Written Examination Content Domain: HEAD & FACE PAIN - Frequency

Distribution of Scores .......................................................................................................... 35

Table 15 - Written Examination Content Domain: BE/BG/LB/LS - Frequency Distribution

of Scores ............................................................................................................................... 35

3

Table 16 - Written Examination Content Domain: BS/CN/CVS – Frequency Distribution of Scores ......................................................................................................................................... 36

4

Table 17 - Written Examination Content Domain: ANS/EN - Frequency Distribution of

Scores ................................................................................................................................... 37

Table 18 - Written Examination Content Domain: NT/R/RS - Frequency Distribution of

Scores ................................................................................................................................... 37 Table 19 - Written Examination Content Domain: SP/PN Frequency Distribution of Scores

..............................................................................................................................................................................38

Table 20 - Written Examination Content Domain: ETHICS - Frequency Distribution of

Scores ................................................................................................................................... 38

Table 21 - Written Examination Content Domain: RED FLAG ISSUES - Frequency

Distribution of Scores .......................................................................................................... 39

Appendix

Appendix A - Preliminary Item Analysis for the Written Examination .................................. 16

Appendix B - Item Analysis Orientation .................................................................................... 24

Appendix C - Written Examination Preliminary Item Analysis Review - Finalization of the

Answer Key .......................................................................................................................... 26

Appendix D - Item Analysis Review Committee ....................................................................... 30

Appendix E – Written Examination Frequency Distribution of Scores by Work Activities...31

Appendix F - Written Examination Frequency Distribution of Scores by Content Domain ..35

Appendix G - Performance Examination Checklist Items ........................................................ 40

Appendix H - Physical Examination - Item Analysis Statistics ................................................ 48

Appendix I - Cases - Item Analysis Statistics ............................................................................. 56

Appendix J - Performance Examination - Frequency Distribution of Scores .......................... 57

5

Introduction

The American Chiropractic Neurology Board, Inc. (ACNB) retained the services of Shoreline

Psychometric Services, LLC. for scoring the October 2015 Diplomate of ACNB Written and

Performance Examinations and providing an analysis of the psychometric properties of the

examinations. This technical report provides a summary of the psychometric properties of

the ACNB 2015 examinations.

Taking and passing the examinations are required for obtaining the Diplomate of ACNB.

ACNB developed the Written and Performance Examinations for administration in October

2015. The 2015 Written Examination consists of 300 multiple-choice questions that was

administered in two parts (i.e., a morning session and an afternoon session), with each part

consisting of 150 test questions. The test questions consist of 4 or 5 options, with one

option identified as the correct answer. Total test score was based on the number of test

questions answered correctly. The examination was administered in October 2015 via

paper-and-pencil testing and candidates marked their responses to the test questions on

Scantron forms; a form for Part I and a form for Part II.

The 2015 Performance Examination consists of a physical examination and a case

(scenario) examination. The physical examination was administered to all candidates and

there were 9 forms of the cases (scenarios) that were assigned to candidates based on test

site location, session (morning or afternoon) and test date. The examinations were scored

using checklists that were completed by trained examiners who observed the candidates

during the performance examination. The physical examination consists of 278 checklist

items and the case (scenario) examinations consists of 7 checklist items that were

completed by two examiners per candidate.

Summary Statistics for the Written Examination

Sixty-six candidates sat for the written examination, which was administered in October

2015. The mean raw score of the written examination was 199.95 (or 66.65%) with a

standard deviation (SD) of 25.08 and standard error of measurement (SEM) of 7.48. The

raw scores ranged from 147 to 251. The pass point was 186 (raw score) and the pass rate

was 76%. Cronbach’s alpha coefficient, which is a measure of internal consistency and a

form of reliability, is .9109 (Cronbach, 1951). This coefficient is high, suggesting strong

internal consistency, and is within an acceptable range. The decision consistency estimate

(Livingston, 1972) was .9323. This estimate indicates strong decision consistency. These

statistical data are displayed in Table 1. A frequency distribution of scores is displayed in

Table 2.

6

Table 1 - Written Examination Summary Statistics

2015 Written Examination

Statistics

Total Number Of Candidates 66

Percentage of Candidates Passing 76%

Passing Point 186

M 199.95

SD 25.08

SEM (at the cut score) 7.48

Decision Consistency Estimate (of P/F decisions) Livingston 0.9323

Reliability Estimate (Cronbach’s Alpha) .9109

Total Number of Items on Exam 300

Table 2 – Written Examination Frequency Distribution of Total Test Scores

Score Frequency Percent Cumulative Percent

147 1 1.52 1.52

148 1 1.52 3.03

149 1 1.52 4.55

150 1 1.52 6.06

152 1 1.52 7.58

164 2 3.03 10.61

165 1 1.52 12.12

166 1 1.52 13.64

167 1 1.52 15.15

169 2 3.03 18.18

177 1 1.52 19.70

182 1 1.52 21.21

184 1 1.52 22.73

185 1 1.52 24.24

186 2 3.03 27.27

187 1 1.52 28.79

190 2 3.03 31.82

191 1 1.52 33.33

193 3 4.55 37.88

196 1 1.52 39.39

197 1 1.52 40.91

198 2 3.03 43.94

199 1 1.52 45.45

201 1 1.52 46.97

202 1 1.52 48.48

203 4 6.06 54.55

206 2 3.03 57.58

7


209 1 1.52 59.09

210 1 1.52 60.61

211 4 6.06 66.67

213 1 1.52 68.18

214 1 1.52 69.70

216 1 1.52 71.21

217 1 1.52 72.73

219 4 6.06 78.79

222 1 1.52 80.30

223 1 1.52 81.82

225 1 1.52 83.33

226 1 1.52 84.85

227 1 1.52 86.36

228 1 1.52 87.88

229 2 3.03 90.91

230 2 3.03 93.94

234 1 1.52 95.45

235 1 1.52 96.97

246 1 1.52 98.48

251 1 1.52 100.00

Total 66 100.00

Table 3 presents the following statistical summary data on Parts I and II of the Written

Examination: highest score, lowest score, median, mean, standard deviation (SD),

reliability, and the standard error of measurement (SEM). The highest and lowest scores on

Parts I and II are similar, as Part I’s scores range from 64 (42.67%) to 127 (84.67%), and

Part II’s scores range from 71 (47.33%) to 125 (83.33%). The mean scores differ by 4.54

points or by 3.03%, as the mean score of Part I is 97.70 (65.13%) and the mean score of

Part II is 102 (68.16%). The SDs for the parts are similar, as the SD for Part I is 13.77 and

the SD for Part II is 12.74. The reliability for Parts I and II are 0.85 and 0.83, respectively,

and the SEMs are 5.36 and 5.19, respectively. The data suggests that the parts of the

examination are comparable in terms of their statistical properties.

Table 3 - Written Examination Performance on Parts I and II

Part I Part II

Number of Examinees 66 66

Number of Items 150 150

Highest Score 127 (84.67%) 125 (83.33%)

Lowest Score 64 (42.67%) 71 (47.33%)

Median 98.50 (65.67%) 104.00 (69.33%)

Mean 97.70 (65.13%) 102.24 (68.16%)

SD 13.77 12.74

Reliability 0.85 0.83

SEM 5.36 5.19

8

Analysis of Subcomponent Data

As shown in Tables 4 and 5, the written examination is composed of the following two

elements: 1) Work Activities and 2) Content Domains. The tables show the number and

percentage of test questions per element and the test specifications.

Table 4 - Written Examination Percent & Number of Items by Work Activities

1. History 3.73% 11 3.73%

2. Physical Exam 7.66% 23 7.66%

3. Special Studies 9.22% 28 9.22% 4. Diagnosing/Diagnoses/Disease Processes/Metabolic

Rate/Pathways 42.60%

128 42.60%

5. Treatment and Rehabilitation (includes anatomy &

physiology pertinent to Treatment & Rehabilitation) 34.99%

105 34.99%

6. Referral 1.80% 5 1.80%

Total 100.00% 300 100.00%

Table 5 - Written Examination Percent & Number of Items by Content Domain

Autonomic Nervous System (ANS) 6.64% 20 6.67%

Brain & Its Environment (BE) 5.73% 17 5.67%

Basal Ganglia (BG) 6.56% 20 6.67%

Brainstem (BS) 5.76% 19 6.33%

Cranial Nerves (CN) 5.87% 17 5.67%

Cerebellum (CVS) 6.43% 19 6.33%

Ethics (E) 2.00% 6 2.00%

Head and Face Pain (HFP) 6.61% 20 6.67%

Lobes of the Brain (LB) 6.42% 19 6.33%

Limbic System (LS) 5.25% 15 5.00%

Neuro-Endocrine System (NE) 4.12% 12 4.00%

Neuron Theory (NT) 5.50% 16 5.33%

Pain (P) 6.51% 21 7.00%

Peripheral Nerves (PN) 5.30% 16 5.33%

Receptor Systems (R) 4.84% 12 4.00%

Work Activities Test

Specifications

Written

Examination

# Of Percent of

Items Items

Content Domain Test Specifications Written Examination

# Of Items Percent of Items

9

Red Flag (RF) 4.51% 14 4.67%

Reflexogenic Systems (RS) 6.56% 21 7.00%

Spinal Cord (SC) 5.40% 16 5.33%

Total 100.01% 300 100.00%

Since the ACNB provides candidates with sub-scores for the components comprising Work

Activities and Content Domains, it was recommended to the ACNB that a review of the

components comprising the Work Activities and Content Domains be conducted for the

purpose of determining whether some of the components are related and could be

combined for the purpose of score reporting. The reason for this recommendation was to

enhance the reliability of the data, as some of the components require a small number of

items, and the reliability of those components would be low. For the work activities, ACNB

combined Special Studies and Referral, as they are related. A review of the components

comprising the Content Domains by ACNB resulted in the following combinations:

1. Head and Face Pain/Pain (HFP/P)

2. Brain and Its Environment/Basal Ganglia/Lobes of the Brain/Limbic System

(BE/BG/LB/LS) 3. Brainstem/Cranial Nerves/Cerebellum (BS/CN/CVS)

4. Autonomic Nervous System/Neuro-Endocrine System (ANS/NE)

5. Neuron Theory/Receptor Systems/Reflexogenic Systems (NT/R/RS)

6. Spinal Cord/Peripheral Nerves (SC/PN)

7. Ethics (E)

8. Red Flag Issues (RF)

The combinations indicated above were used to produce score report data and are

described and presented in this section of the report. Tables 6 and 7 present the following

statistics for each subcomponent of the Work Activities and Content Domains: number of

items, minimum score, maximum score, mean (M), standard error of the mean (SE),

standard deviation (SD), variance, reliability, and the SEM.

For the Work Activities, the reliability indices for the subcomponents range from 0.24 for

History to 0.74 for Diagnosis. The SEM ranged from 2.08 for Physical Examination to 4.77

for Diagnosis. (See Table 6 for these data.) Appendix D presents the frequency

distributions of test scores by Work Activity.

For the Content Domains, the reliability indices for the components ranged from .17 for

Ethics to .71 for BE/BG/LB/LS. (See Table 7 for these data.) Appendix E presents the

frequency distributions of test scores by Content Domain.

Content Domain Test Specifications Written Examination

# Of Items Percent of Items

10

Table 6 - Written Examination - Work Activities Sub-Score Statistics

Diagnosis 66 128 88.95 1.14 9.29 66 108 .7359 4.77

History 66 11 6.26 0.41 3.30 2 10 .2377 2.88

Physical

Examination 66 23 14.50 0.01 2.74 8 20 .4243 2.08

Special Studies &

Referral 66 33 20.62 0.37 3.02 15 27 .3201 2.49

Treatment/

Rehabilitation 66 105 69.64 0.87 7.09 55 86 .5921 4.53

Table 7 - Written Examination - Content Domain Sub-Score Statistics

N of Candidates 66 66 66 66 66 66 66 66

N of Items 41 71 55 32 49 32 6 14

Minimum 14.00 35.00 14.00 12.00 17.00 12.00 3.00 5.00

Maximum 32.00 64.00 51.00 28.00 41.00 33.00 6.00 14.00

Mean 23.45 51.08 34.85 20.89 31.35 23.83 4.53 9.95

SE 0.49 0.85 0.90 0.49 0.60 0.51 0.11 0.26

SD 4.00 6.94 7.30 3.98 4.88 4.11 0.93 2.15

Variance 16.04 48.10 53.33 15.82 23.80 16.91 0.87 4.63

Reliability (Spearman) .5829 .7076 .6520 .5217 .6254 .5217 .1698 .3231

SEM 2.58 3.75 4.31 2.75 2.99 2.84 .85 1.77

Overview of Item Analysis Statistics

The item analysis includes difficulty and discrimination statistics. These statistics provide

information about the relative difficulty level of test questions and how well test questions

perform in relationship to total test score (i.e., discrimination). It is recommended that

item analysis statistics be used as a guide to help subject matter experts in their review of

test questions when finalizing the answer key and for future revision of the examination.

P-values indicate the difficulty level of the test questions. It is the proportion of candidates

selecting a test question’s option. P-values may range from 0.00 to 1.00. P-values are

produced for each option of a test question. As a general rule, test questions in which the p-

value for the correct answer is 0.90 or higher indicates that the majority of candidates (90%

or greater) selected the correct answer. Test questions with p-values of 0.90 or greater

are considered easy test questions. In contrast, test questions with p-values of 0.30

Candidates

M SE SD Max

SEM

10

or lower indicate that a small number of candidates (30% or less) selected the correct

answer. Test questions with low p-values for the correct answer are considered to be

difficult test questions. Subject matter experts should review test questions with low p-

values (e.g., 0.30 or less) to rule out the possibility that there is more than one correct

answer, or that the question is mis-keyed (e.g., option “A” was keyed as the correct answer,

but the actual correct answer is option “D”). If subject matter experts rule out problems

with the correctness of the answer key, they may need to assess the clarity of the question’s

wording and/or the appropriateness of the test question for the examination (e.g., the

subject matter was accurate but the question was not appropriate for the purpose of the

examination). P-values were produced for both forms.

A discrimination statistic was produced for test questions on the written examination. The

discrimination statistic (i.e., point bi-serial correlation) shows the relationship of

performance on the test question and total test score. The statistic is “corrected item-total

correlation” because the score on the individual test question is not included in the total

score when calculating the correlation. If the test question score was included in the

calculation, the correlation would be inflated. The statistic, which is a correlation, may

range from -1.00 to +1.00. It is desirable to have correlations that are at least 0.10 or higher

(0.20 would be desirable), as the correlation indicates a positive relationship between

performance on the test question and total test score. Test questions with correlations

less than 0.10 suggest a poor or inverse relationship between performance on the

test question and total test score. Subject matter experts should review these questions

to verify their accuracy and clarity, and the accuracy of the questions’ answer keys.

These data were produced, reviewed and items with undesirable statistical properties as

described above (i.e., items with low p-values and low discrimination indices) were flagged

for review by subject matter experts to finalize the answer key. See Appendix A for the

preliminary item analysis. A complete preliminary item analysis (i.e., data for all options)

was provided to ACNB separately.

On November 12, 2015, a meeting was held via web conference to review the test questions

that were flagged for review. In advance of the meeting, the ACNB provided the committee

members with password-protected access to the examination, a description of item analysis

data and the purpose of the review (See Appendix B – Item Analysis Orientation), and

a list of items that were flagged for review. The meeting resulted in altering the

answer key to some of the questions as well as flagging questions for future review and

possibly revision. See Appendix C for the list of items that were flagged for review and the

final answer key changes for the examination. See Appendix D for the list of committee

members. The complete preliminary item analysis and the final item analysis were

provided to ACNB separately.

11

Summary Statistics for the Performance Examination

The performance examinations were scored using checklists that were completed by

trained examiners who observed the candidates during the performance examination. The

physical examination consisted of 278 checklist items and the case (scenario) examinations

consisted of 7 checklist items that were completed by two examiners per candidate. (See

Appendix F for a list of the physical examination and case items on the 2015 Performance

Examination.) Seventy-five (75) candidates took the performance examination.

The approach for scoring the performance examination was based information that the

ACNB provided Shoreline Psychometric Services, LLC. for use in scoring the examination.

To score the examination, the number of checked items was summed to derive the

candidates’ raw scores for the physical examination and for the case. The two sets of raw

scores (i.e., the scores resulting from the two examiners’ checklists) were averaged (i.e., an

average score for the physical examination and an average score for the case). The Physical

Examination is weighted as 50% and the Cases as 50%. The cut score that was used for the

2014 Physical Examination was applied, as that examination was not amended by the ACNB

for the 2015 test administration. Since the cases were developed for the 2015 test

administration, passing point studies were conducted in December 2015 to derive a passing

score for each case. (See the 2015 Pass Point Study Report for details.) The final cut

score for each performance examination is derived from the weighted cut score (i.e., the

physical examination pass point and the case pass point).

Overview of Item Analysis Statistics

As a result of reviewing summary data of the performance on the examination (i.e.,

percentage correct on the checklist items on the physical examination and the percentage

correct on the checklist items on the cases), the ACNB did not alter scores (i.e., akin to

altering the answer key on a written examination). See Appendix H for the Item Analysis

for the Physical Examination and Appendix I for the Item Analysis for the Cases. It should

be noted that prior to viewing summary data, ACNB instructed Shoreline Psychometric

Services, LLC. to give candidates credit on the Review & Clarification of Prescreen History

checklist item for the Cases. The reason for giving credit is that the ACNB decided that

there was sufficient information provided in the cases that would not necessitate the

candidate having to ask the examiners for clarification, as the ACNB wanted to maintain

standardization of the test administration by not having the examiners respond to a

candidate’s request for additional information.

Nine cases were administered during the October 2015 test administration. As shown in

Table 8 below, there were small numbers of candidates taking each case, as the number of

candidates per case ranged from 3 candidates for Case #2 to 12 candidates for Case #1.

Therefore, the statistics for these examinations should be interpreted with caution, as they

are based on small numbers of candidates. ACNB indicated that since the examination is

administered in three locations, over a two-day period in morning and afternoon sessions,

12

to ensure the security of an examination (i.e., case), the cases were assigned to candidates

based on location, test date, and session.

The data in Table 8 show that the pass rate was high for the examination, as only one of the

75 candidates taking the performance examination failed the examination. The total mean

(weighted) scores ranged from 70.32% for Case #9 to 98.66% for Case #11. The table also

presents the weighted pass point for each examination, the weighted total score standard

deviation (SD), the weighted total standard error of measurement (SEM), the mean score

(raw) for the physical examination (PE), the SD (raw), and the mean (raw) scores and SDs

for the cases. The reliability indices and decision consistency indices for all examinations

were .99, respectively. Appendix J presents a frequency distribution of total scores.

Table 8 - Performance Examination Statistics

Performance Examination Statistics

Statistics

Physical Exam and Case #

#1 #2 #3 #4 #5 #7 #9 #10 #11

Total # Of

candidates

% Of Candidates

Passing

Passing Point

(Percent of

Weighted Score)

Total Mean Score

(Weighted)

Total Score SD

(Weighted)

Total Score SEM

at the Cut Score

(Weighted)

PE Mean Score

(Raw)

PE SD (Raw)

Case Mean Score

(Raw)

Case SD (Raw)

Decision

Consistency

Estimate (of P/F

decisions)

Livingston

Reliability

Estimate (Inter-

rater reliability -

Pearson

Correlation) Total Number of

12 3 9 10 17 4 5 7 8

100% 100% 100% 100% 100% 100% 87.5% 100% 100%

63%

62%

64%

64%

63%

62%

65%

65%

60%

95.05% 89.08% 94.72% 93.40% 94.35% 89.83% 70.32% 91.63% 98.66%

4.63 9.82 4.42 10.44 6.69 6.22 19.73 9.08 1.75

4.83

4.85

4.80

4.80

4.83

4.85

4.77

4.77

4.90

264.38 257.00 268.50 263.15 260.59 231.38 192.40 268.36 275.50

13.52 15.26 5.39 14.09 14.90 26.97 46.92 4.73 2.17

6.54 6.00 6.50 6.45 6.65 6.75 5.00 6.07 6.88

0.66 1.00 0.62 1.42 0.68 0.46 1.87 1.27 0.23

.99

.99

.99

.99

.99

.99

.99

.99

.99

.99 .99 .99 .99 .99 .99 .99 .99 .99

Items on Exam 285 285 285 285 285 285 285 285 285

13

Quality Control Checks of the Data

The responses to the written examination were completed by candidates on Scantron

forms and Shoreline Psychometric Services, LLC. scanned and scored the examinations.

Since the written examination was administered in two parts, each candidate completed

two Scantron forms: a form for Part I, and a form for Part II. Each Scantron form was

checked against the candidates’ rosters to ensure that all forms were received. The

Scantron forms were checked for completion (i.e., candidates’ name, ID number) and for

stray markings that would be flagged by the scoring software. To check the accuracy of

scoring process by the scoring software, several Scantron forms were hand scored,

including those that scored near the pass point. To further ensure the accuracy of scoring,

a new scoring file was created and the Scantron forms were scanned for a second time and

the scoring was checked against the original score file.

The candidates’ responses to the performance examinations were marked on checklists

used by examiners who observed candidates’ performance. There were two checklists

used by the examiners: 1) a physical examination checklist, and 2) a checklist for the case.

Two examiners observed each candidate; therefore, there were four checklists per

candidate. Each checklist form was checked against the candidate roster to ensure that all

checklists were received. The data on the checklists were entered into an Excel

spreadsheet. Checklists in which the markings required verification (i.e., hard to read)

were sent to the ACNB for reconciliation. After entry of the checklist data into the Excel

spreadsheet, several candidates’ checklists were rescored and the scores were checked

against the original scores. The score data were provided to the ACNB in Excel

spreadsheets, as the ACNB prepared the candidates’ individual score reports and sent them

to the candidates.

Summary and Recommendations

Overall, the written examination summary statistics indicate that the examination has

reliability and the decision consistency indices within acceptable ranges for certification

examination programs. The statistics for the subcomponents show low to moderate

reliability indices, which is typical of subcomponents of most certification examinations, as

there are smaller numbers of test questions among the components and reliability is

sensitive to test length. When reporting sub-score performance data to candidates, it is

recommended that an explanation of the reliability of the data be provided, as performance

on the sub-areas are based on small numbers of items.

The item analysis for the written examination resulted in identifying test questions that

have low p-values (i.e., difficult) and/or have poor discrimination. Subject matter experts

reviewed these data and the test questions to finalize the examination’s answer key. As

described previously, it is recommended that item analysis statistics be used as a guide to

help subject matter experts in their review of test questions and revision of the

examination. In addition to reviewing test questions with low p-values and poor

14

discrimination indices, test questions that have high p-values should be reviewed when

updating examination forms, as such test questions do not discriminate well between high

and low performers. As with difficult test questions, it is recommended that easy questions

be revised or replaced when updating examinations because they do not distinguish

between those who have mastered knowledge from those who have not.

Akin to the written examination, the summary statistics and item analyses for the

performance examination should be taken into consideration when reviewing and

updating the examination for 2016. However, given the small numbers of candidates

taking each examination by case, the statistical summary results for the cases should be

reviewed with caution.

As with any certification examination program, the performance of examinations should be

monitored on an ongoing basis, and the statistical data presented in this report should be

considered when assessing the quality of examinations and developing new examinations.

15

References

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika,

16, 297-334.

Livingston, S. A. (1972). Criterion-referenced applications of classical test theory. Journal of

Educational Measurement, 13-2.

16

Appendix A - Preliminary Item Analysis for the Written Examination

1 0.48 0.34

2 0.89 0.07

3 0.67 0.46

4 0.41 0.22

5 0.68 0.21

6 0.77 0.52

7 0.73 0.15

8 0.50 0.12

9 0.62 0.31

10 0.79 0.16

11 0.91 0.16

12 0.83 0.19

13 0.62 0.24

14 0.76 0.13

15 0.82 0.20

16 0.88 0.25

17 0.74 0.34

18 0.89 0.09

19 0.50 0.38

20 0.55 -0.25

21 0.88 0.32

22 0.80 0.15

23 0.58 0.15

24 0.41 0.13

25 0.74 0.38

26 0.48 -0.14

27 0.47 0.23

28 0.77 0.29

29 0.59 0.53

30 0.56 0.36

31 0.55 0.25

32 0.71 0.40

33 0.38 0.49

34 0.70 0.57

35 0.56 0.14

36 0.55 0.37

Written Examination Preliminary Item Analysis

Point Biserial Correlation

17

Item# p-value Point Biserial Correlation

37 0.61 0.37

38 0.42 0.06

39 0.67 0.03

40 0.79 0.19

41 0.53 0.29

42 0.52 0.05

43 0.86 0.36

44 0.77 0.31

45 0.94 0.27

46 0.55 0.24

47 0.88 0.23

48 0.65 0.13

49 0.94 0.18

50 0.94 0.04

51 0.86 0.39

52 0.61 0.32

53 0.70 0.16

54 0.55 0.31

55 0.85 0.09

56 0.48 -0.08

57 0.82 0.29

58 0.76 0.37

59 0.53 0.35

60 0.86 0.29

61 0.70 0.00

62 0.48 0.05

63 0.67 0.17

64 0.82 0.40

65 0.52 0.06

66 0.45 0.05

67 0.62 0.16

68 0.59 0.27

69 0.70 0.17

70 0.86 0.33

71 0.88 0.09

72 0.58 0.31

73 0.73 0.10

74 0.83 0.18

75 0.85 -0.10



Item#

76

77

78

p-value

0.55

0.53

0.20


105

106

107

108

109

110

111

112

113

114

0.47

0.38

0.92

0.71

0.35

0.92

0.68

0.50

0.70

0.97

0.02

0.28

0.09

0.10

0.18

0.13

0.14

0.16

0.37

0.08

18

79 0.20 -0.08

80 0.68 0.25

81 0.58 0.33

82 0.32 0.42

83 0.77 0.22

84 0.27 0.01

85 0.68 0.42

86 0.27 0.30

87 0.89 -0.05

88 0.70 0.50

89 0.74 0.40

90 0.70 0.19

91 0.76 0.34

92 0.17 0.05

93 0.88 0.28

94 0.27 -0.14

95 0.50 0.07

96 0.74 0.16

97 0.26 0.31

98 0.86 0.26

99 0.89 0.23

100 0.53 0.16

101 0.79 0.22

102 0.47 0.05

103 0.44 0.25

104 0.36 0.09


Item#

115

116

117

118

119

120

121

122

p-value

0.33

0.67

0.65

0.53

0.74

0.50

0.65

0.50


146

147

148

149

150

151

152

153

0.62

0.15

0.48

0.48

0.74

0.53

0.74

0.65

0.42

-0.16

0.00

0.02

0.00

0.33

0.27

0.30

19

123 0.45

124 0.74

125 0.91

126 0.61

127 0.48

128 0.53

129 0.85

130 0.47

131 0.58

132 0.53

133 0.83

134 0.58

135 0.76

136 0.76

137 0.71

138 0.21

139 0.76

140 0.80

141 0.48

142 0.76

143 0.73

144 0.73

145 0.41


Item#

154

155

156

157

158

159

160

161

p-value

0.71

0.53

0.65

0.68

0.64

0.68

0.67

0.73


190

191

192

0.74

0.97

0.39

-0.07

0.13

0.23

20

162 0.80

163 0.64

164 0.26

165 0.64

166 0.77

167 0.53

168 0.67

169 0.53

170 0.82

171 0.68

172 0.91

173 0.73

174 0.71

175 0.23

176 0.73

177 0.64

178 0.55

179 0.89

180 0.86

181 0.79

182 0.64

183 0.50

184 0.89

185 0.86

186 0.53

187 0.94

188 0.76

189 0.89


Item#

193

194

195

196

197

198

199

200

p-value

0.67

0.29

0.85

0.88

0.83

0.92

0.71

0.80


230

231

0.44

0.56

0.47

-0.03

21

201 0.53

202 0.91

203 0.24

204 0.86

205 0.56

206 0.48

207 0.70

208 0.91

209 0.83

210 0.59

211 0.47

212 0.95

213 0.79

214 0.86

215 0.52

216 0.83

217 0.70

218 0.71

219 0.76

220 0.77

221 0.52

222 0.91

223 0.92

224 0.76

225 0.68

226 0.79

227 0.79

228 0.32

229 0.39


Item#

232

233

234

235

236

237

238

239

p-value

0.45

0.50

0.22

2

0.88

0.47

0.33

0.44

0.68


22

240 0.39 0.39

241 0.33 0.35

242 0.44 0.23

243 0.64 0.14

244 0.77 0.10

245 0.82 0.00

246 0.80 0.07

247 0.56 0.21

248 0.74 -0.04

249 0.35 -0.06

250 0.62 0.11

251 0.76 0.20

252 0.39 0.13

253 0.58 0.29

254 0.21 0.11

255 0.62 0.37

256 0.76 0.14

257 1.00 0.00

258 0.77 0.32

259 0.91 0.35

260 0.58 0.08

261 0.09 -0.29

262 0.58 0.42

263 0.17 0.03

264 0.65 0.38

265 0.44 0.30

266 0.91 0.29

267 0.74 0.03

268 0.15 -0.28

269 0.83 0.11

270 0.71 0.24

23

Item# p-value Point Biserial Correlation

271 0.74 0.19

272 0.70 0.27

273 0.71 0.27

274 0.52 0.00

275 0.94 0.25

276 0.88 0.42

277 0.94 0.11

278 1.00 0.00

279 0.77 0.40

280 0.80 -0.04

281 0.92 0.40

282 0.50 -0.04

283 0.64 0.27

284 0.88 0.35

285 0.50 0.30

286 0.74 0.21

287 0.48 0.18

288 0.76 0.51

289 0.56 -0.12

290 0.92 0.11

291 0.80 0.30

292 0.76 0.06

293 0.12 0.27

294 0.74 0.37

295 0.79 0.06

296 0.77 0.41

297 0.77 0.31

298 0.27 -0.14

299 0.86 0.14

300 0.52 0.18

Mean 0.65 0.20

SD 0.20 0.16

MIN 0.09 -0.29

MAX 1.00 0.57


24

Appendix B - Item Analysis Orientation

INTRODUCTION TO ITEM ANALYSIS STATISTICS

The item analysis includes difficulty and discrimination statistics. These statistics provide

information about the relative difficulty level of test questions and how well test questions

perform in relationship to total test score (i.e., discrimination). It is recommended that item

analysis statistics be used as a guide to help subject matter experts in their review of test

questions when finalizing answer keys and revising exam forms.

DIFFICULTY: P---values indicate the difficulty level of the test questions. It is the

proportion of candidates selecting a test question’s option. As a general rule, test questions

with p--- values of 0.30 or lower indicate that a small number of candidates (30% or less)

selected the correct answer. Test questions with low p---values are considered to be

difficult test questions.

1) Test questions with low p---values (e.g., 0.30 or less) should be reviewed to rule out the

possibility that there is more than one correct answer, or that the question is mis-

keyed (e.g., option “A” was keyed as the correct answer, but the actual correct

answer is option “D”).

2) If there is no problem with the correctness of the answer key, then assess the

appropriateness of the test question for the examination (e.g., the subject matter

was accurate but the question was not appropriate for the purpose of the

examination). DISCRIMINATION: The point biserial correlation is a discrimination

statistic that shows the relationship of performance on the test question and total

test score. The statistic, which is a correlation, may range from ---1.00 to +1.00. It is

desirable to have correlations that are at least 0.10 or higher (0.20 would be

desirable), as the correlation indicates a positive relationship between performance

on the test question and total test score. Test questions with correlations less than

0.10 should be reviewed by subject matter experts to verify their accuracy, clarity

and the questions’ answer keys, as such a low statistic suggests a poor or inverse

relationship between performance on the test question and total test score.

INSTRUCTIONS: The test questions listed in the tables below were flagged for

review because they have item analysis statistics indicating that they are difficult

questions and/or they have poor discrimination properties. Please review the test

questions to determine the following:

1) The item was mis-keyed (e.g., A was indicated as the correct answer, but C is correct).

2) The item has more than one correct answer (e.g., A and D are correct).

3) The item was unclear; therefore, a correct answer could not be identified.

4) The item was clear, appropriate for the examination, and the keyed answer is correct

(e.g., item is “ok as is”).

25

As you review the test questions listed in the tables below, you may wish to indicate the

above-mentioned determinations in the comments sections of the tables. Please have your

comments available for the web--conference.

26

Appendix C - Written Examination Preliminary Item Analysis Review - Finalization of

the Answer Key

PRELIMINARY ITEM ANALYSIS

ITEMS THAT ARE FLAGGED FOR REVIEW

PART ONE WRITTEN EXAMINATION

Item # p-value Comments

78 0.20 KEY A, B, C, D AS CORRECT, FLAG FOR REVIEW

79 0.20 OK

82 0.32 OK

84 0.27 KEY A AND B AS CORRECT, FLAG FOR REVIEW

86 0.27 OK

92 0.17 OK

94 0.27 OK

97 0.26 OK

104 0.36 OK

109 0.35 OK

115 0.33 OK

138 0.21 OK

147 0.15 KEY A, B, C, D AS CORRECT – FLAG FOR REVIEW

2 0.074 OK

18 0.092 OK BUT FLAG FOR REVIEW

20 -0.249 OK

26 -0.138 OK

38 0.062 OK

39 0.033 OK

42 0.053 OK

50 0.037 OK

55 0.093 OK

56 -0.082 OK

61 0.000 OK

62 0.046 OK

65 0.060 OK

PART ONE – Items with Low p-values (n=13)

DIFFICULT ITEMS

ITEM #

PART ONE – Items with Low Point Biserial Correlations (p < .10) (n=42)

ITEMS WITH POOR DISCRIMINATION

Point Biserial

Correlation Comments

27

66 0.050 OK

71 0.085 OK

75 -0.099 OK

*78 -0.223 SEE PREVIOUS SECTION


*84 0.007 SEE PREVIOUS SECTION

87 -0.054 OK



95 0.067 OK

102 0.049 OK


105 0.018 OK

107 0.086 OK

108 0.096 OK

114 0.078 OK

120 0.092 OK

126 0.089 OK

129 0.080 OK

130 -0.059 OK

131 -0.071 OK

132 -0.025 OK

134 -0.007 OK


145 -0.053 OK BUT FLAG FOR REVIEW

*147 -0.158 SEE PREVIOUS SECTION - FLAG FOR REVIEW



150 -0.003 OK BUT FLAG FOR REVIEW

*Note: These items were flagged as very difficult.

PART TWO WRITTEN EXAMINATION

ITEM #

PART ONE – Items with Low Point Biserial Correlations (p < .10) (n=42)


Point Biserial


PART TWO - Items with low p-values (n=14)

DIFFICULT ITEMS

28

Item # Comments

14 0.26 KEY A AND D AS CORRECT – FLAG CASE

25 0.23 OK

44 0.29 OK

53 0.24 OK

78 0.32 OK

87 0.33 OK

91 0.33 OK

99 0.35 KEY B AND D AS CORRECT – FLAG CASE

104 0.21 KEY B AND C AS CORRECT – FLAG CASE

111 0.09 THE CORRECT ANSWER IS B NOT D, B IS CORRECT NOT D

113 0.17 OK

118 0.15 KEY C AND D AS CORRECT

143 0.12 OK

148 0.27 OK

11 0.06 OK

13 -0.02 OK

*14 0.04 SEE PREVIOUS SECTION – DOUBLE KEY

24 0.02 OK


29 0.04 OK

39 -0.04 OK

40 -0.07 OK

48 0.07 OK


55 -0.03 OK

59 0.05 OK

61 -0.04 OK

62 -0.04 OK

66 0.08 OK

67 0.07 OK

68 -0.04 KEY A AND C AS CORRECT

70 0.07 OK

81 -0.03 OK

ITEM #

PART TWO – Items with Low Point Biserial Correlations (p < .10) (n=40)


Point Biserial


29

86 0.01 OK

30

88 0.01 OK

94 0.10 KEY A AND D AS CORRECT

95 0.00 OK

96 0.07 KEY A, B, C, D, AS CORRECT

98 -0.04 OK

*99 -0.06 KEY B AND D AS CORRECT

107 0.00 OK

110 0.08 OK



117 0.03 OK


124 0.00 OK

128 0.00 OK

130 -0.04 OK

132 -0.04 OK

139 -0.12 OK

142 0.06 OK

145 0.06 OK


*Note: These items were flagged as very difficult.

ITEM #

PART TWO – Items with Low Point Biserial Correlations (p < .10) (n=40)


Point Biserial


30

Appendix D - Item Analysis Review Committee

Item Analysis Review Committee First Name Last Name Location

1. Dr. Gail Henry TX

2. Dr. Daniel Demian CN

3. Dr. Karla Mehlenbacher CA

4. Dr. Ed Barrington AK

5. Dr. George Petryk FL

6. Dr. Jessica Gundelfinger NC

7. Dr. Darcy Dane NC

8. Dr. Michaele Posey CO

9. Dr. Erin Van Veldhuizen TX

10. Dr. Candace Duty OH

11. Dr. David Aguilar NY

31

Appendix E – Written Examination Frequency Distribution of Scores by Work

Activities

Table 9 – Written Examination Work Activity: Diagnosis – Frequency Distribution of

Scores

Written Examination Work Activity

Diagnosis Frequency Distribution

Score Frequency Percent Cumulative

Percent

56 1 1.52 1.52

57 1 1.52 3.03

62 1 1.52 4.55

63 1 1.52 6.06

67 1 1.52 7.58

68 1 1.52 9.09

69 1 1.52 10.61

70 1 1.52 12.12

71 1 1.52 13.64

73 1 1.52 15.15

74 1 1.52 16.67

76 1 1.52 18.18

77 1 1.52 19.70

78 1 1.52 21.21

79 3 4.55 25.76

81 1 1.52 27.27

82 2 3.03 30.30

83 3 4.55 34.85

84 1 1.52 36.36

85 3 4.55 40.91

87 3 4.55 45.45

88 1 1.52 46.97

89 1 1.52 48.48

90 1 1.52 50.00

91 2 3.03 53.03

92 1 1.52 54.55

93 1 1.52 56.06

94 2 3.03 59.09

95 2 3.03 62.12

96 6 9.09 71.21

97 3 4.55 75.76

98 1 1.52 77.27

99 2 3.03 80.30

100 2 3.03 83.33

101 1 1.52 84.85

102 3 4.55 89.39

103 1 1.52 90.91

105 2 3.03 93.94

106 1 1.52 95.45

32


Diagnosis Frequency Distribution


Percent

107 2 3.03 98.48

109 1 1.52 100.00

Total 66 100.00

Table 10 - Written Examination Work Activity: HISTORY - Frequency Distribution of

Scores


History Frequency Distribution


Percent

2 2 3.03 3.03

4 7 10.61 13.64

5 15 22.73 36.36

6 12 18.18 54.55

7 13 19.70 74.24

8 12 18.18 92.42

9 4 6.06 98.48

10 1 1.52 100.00

Total 66 100.00

Table 11 - Written Examination Work Activity: PHYSICAL EXAM - Frequency

Distribution of Scores


Physical Exam Frequency Distribution


Percent

8 1 1.52 1.52

9 2 3.03 4.55

10 4 6.06 10.61

11 8 12.12 22.73

12 5 7.58 30.30

13 3 4.55 34.85

14 8 12.12 46.97

15 7 10.61 57.58

16 10 15.15 72.73

17 7 10.61 83.33

18 6 9.09 92.42

19 1 1.52 93.94

20 3 4.55 98.48

21 1 1.52 100.00

Total 66 100.00

33

Table 12 - Written Examination Work Activity: SPECIAL STUDIES & REFERRAL -

Frequency Distribution of Scores


13 1 1.52 1.52

14 3 4.55 6.06

15 1 1.52 7.58

16 2 3.03 10.61

17 5 7.58 18.18

18 5 7.58 25.76

19 5 7.58 33.33

20 9 13.64 46.97

21 6 9.09 56.06

22 10 15.15 71.21

23 7 10.61 81.82

24 4 6.06 87.88

25 5 7.58 95.45

26 1 1.52 96.97

27 2 3.03 100.00

Total 66 100.00

Table 13 - Written Examination Work Activity: TREATMENT/REHABILITATION -

Frequency Distribution of Scores

Written Examination - Work Activity

Treatment/Rehabilitation Frequency Distribution


Percent

47 1 1.52 1.52

50 3 4.55 6.06

55 1 1.52 7.58

58 2 3.03 10.61

59 2 3.03 13.64

60 2 3.03 16.67

61 3 4.55 21.21

62 1 1.52 22.73

63 2 3.03 25.76

65 1 1.52 27.27

66 1 1.52 28.79

67 4 6.06 34.85

68 2 3.03 37.88

69 2 3.03 40.91

70 4 6.06 46.97

71 1 1.52 48.48

72 3 4.55 53.03

73 4 6.06 59.09

74 2 3.03 62.12

75 3 4.55 66.67

34

Written Examination - Work Activity

Treatment/Rehabilitation Frequency Distribution


Percent

76 5 7.58 74.24

77 4 6.06 80.30

78 1 1.52 81.82

79 1 1.52 83.33

80 1 1.52 84.85

81 3 4.55 89.39

83 2 3.03 92.42

84 2 3.03 95.45

87 1 1.52 96.97

89 1 1.52 98.48

91 1 1.52 100.00

Total 66 100.00

35

Appendix F - Written Examination Frequency Distribution of Scores by Content

Domain

Table 14 - Written Examination Content Domain: HEAD & FACE PAIN - Frequency



14 1 1.52 1.52

16 2 3.03 4.55

17 3 4.55 9.09

18 2 3.03 12.12

19 5 7.58 19.70

20 2 3.03 22.73

21 5 7.58 30.30

22 6 9.09 39.39

23 5 7.58 46.97

24 7 10.61 57.58

25 7 10.61 68.18

26 7 10.61 78.79

27 3 4.55 83.33

28 3 4.55 87.88

29 5 7.58 95.45

30 1 1.52 96.97

31 1 1.52 98.48

32 1 1.52 100.00

Total 66 100.00

Table 15 - Written Examination Content Domain: BE/BG/LB/LS - Frequency


35 2 3.03 3.03

37 2 3.03 6.06

40 1 1.52 7.58

41 2 3.03 10.61

42 2 3.03 13.64

43 1 1.52 15.15

44 1 1.52 16.67

45 3 4.55 21.21

46 1 1.52 22.73

47 6 9.09 31.82

48 1 1.52 33.33

49 2 3.03 36.36

Frequency Distribution

Score Frequency Percent

36

50 3 4.55 40.91

51 8 12.12 53.03

52 1 1.52 54.55

53 2 3.03 57.58

54 2 3.03 60.61

55 7 10.61 71.21

56 5 7.58 78.79

57 2 3.03 81.82

58 2 3.03 84.85

59 2 3.03 87.88

60 5 7.58 95.45

62 2 3.03 98.48

64 1 1.52 100.00

Total 66 100.00

Table 16 - Written Examination Content Domain: BS/CN/CVS – Frequency


14 1 1.52 1.52

18 1 1.52 3.03

22 2 3.03 6.06

24 1 1.52 7.58

25 2 3.03 10.61

26 1 1.52 12.12

28 5 7.58 19.70

29 3 4.55 24.24

30 3 4.55 28.79

31 3 4.55 33.33

32 1 1.52 34.85

33 4 6.06 40.91

34 2 3.03 43.94

35 2 3.03 46.97

36 5 7.58 54.55

37 5 7.58 62.12

38 2 3.03 65.15

39 5 7.58 72.73

40 4 6.06 78.79

41 3 4.55 83.33

42 2 3.03 86.36

43 3 4.55 90.91

44 2 3.03 93.94

45 2 3.03 96.97

51 2 3.03 100.00

Total 66 100.00

Frequency Distribution


37

Table 17 - Written Examination Content Domain: ANS/EN - Frequency Distribution

of Scores


12 1 1.52 1.52

13 1 1.52 3.03

14 4 6.06 9.09

15 2 3.03 12.12

16 3 4.55 16.67

17 2 3.03 19.70

18 7 10.61 30.30

19 3 4.55 34.85

20 2 3.03 37.88

21 5 7.58 45.45

22 14 21.21 66.67

23 6 9.09 75.76

24 5 7.58 83.33

25 1 1.52 84.85

26 6 9.09 93.94

27 1 1.52 95.45

28 3 4.55 100.00

Total 66 100.00

Table 18 - Written Examination Content Domain: NT/R/RS - Frequency Distribution

of Scores

17 1 1.52 1.52

23 2 3.03 4.55

25 5 7.58 12.12

27 5 7.58 19.70

28 4 6.06 25.76

29 8 12.12 37.88

30 8 12.12 50.00

31 4 6.06 56.06

32 2 3.03 59.09

33 6 9.09 68.18

34 6 9.09 77.27

35 1 1.52 78.79

36 3 4.55 83.33

37 3 4.55 87.88

39 3 4.55 92.42

40 4 6.06 98.48

41 1 1.52 100.00


38

Total 66 100.00

Table 19 - Written Examination Content Domain: SP/PN Frequency Distribution of

Scores

12 1 1.52 1.52

15 1 1.52 3.03

16 2 3.03 6.06

17 1 1.52 7.58

18 1 1.52 9.09

19 3 4.55 13.64

20 5 7.58 21.21

21 7 10.61 31.82

22 2 3.03 34.85

23 3 4.55 39.39

24 8 12.12 51.52

25 4 6.06 57.58

26 11 16.67 74.24

27 6 9.09 83.33

28 5 7.58 90.91

29 2 3.03 93.94

30 2 3.03 96.97

31 1 1.52 98.48

33 1 1.52 100.00

Total 66 100.00

Table 20 - Written Examination Content Domain: ETHICS - Frequency Distribution of

Scores


3 10 15.15 15.15

4 21 31.82 46.97

5 25 37.88 84.85

6 10 15.15 100.00

Total 66 100.00


39

Table 21 - Written Examination Content Domain: RED FLAG ISSUES - Frequency



5 1 1.52 1.52

6 4 6.06 7.58

7 6 9.09 16.67

8 7 10.61 27.27

9 5 7.58 34.85

10 13 19.70 54.55

11 13 19.70 74.24

12 11 16.67 90.91

13 4 6.06 96.97

14 2 3.03 100.00

Total 66 100.00

40

Appendix G - Performance Examination Checklist Items

1 COGNITIVE Orientation Person

2 COGNITIVE Orientation Place

3 COGNITIVE Orientation Time

4 COGNITIVE Memory Recent

5 COGNITIVE Memory Remote

6 COGNITIVE Memory Mono

7 COGNITIVE Memory Singsong

8 COGNITIVE Cognitive Spell forward/backward World

9 COGNITIVE Cognitive Count back by 7 from 100

10 COGNITIVE Cognitive Dog/Hat/Chair 2min

11 COGNITIVE I I

12 COGNITIVE Blind Spot R

13 COGNITIVE Blind Spot L

14 CRANIAL NERVES ID Coffee Y

15 CRANIAL NERVES II Peripheral Vision RL II Peripheral Vision RL

16 CRANIAL NERVES CN II Peripheral Vision RL CN II- 3 14" R

17 CRANIAL NERVES CN II Peripheral Vision RL CN II- 3 14" L

18 CRANIAL NERVES CN II Peripheral Vision RL CN II- 2 14" Both

19 CRANIAL NERVES V Pain/Pinwheel V1R

20 CRANIAL NERVES V Pain/Pinwheel CN V Pain-V1L

21 CRANIAL NERVES V Pain/Pinwheel CN V Pain-V2R

22 CRANIAL NERVES V Pain/Pinwheel CN VPain-V2L

23 CRANIAL NERVES V Pain/Pinwheel CN VPain-V3R

24 CRANIAL NERVES V Pain/Pinwheel CN VPain-V3L

25 CRANIAL NERVES Lt Touch CN V-Lt-V1R

26 CRANIAL NERVES Lt Touch CN V-LtV1L

27 CRANIAL NERVES Lt Touch CN V-LtV2R


29 CRANIAL NERVES Lt Touch CN V-LtV3R


31 CRANIAL NERVES Onion Skin Pattern R

32 CRANIAL NERVES Onion Skin Pattern L

33 CRANIAL NERVES VII Motor

34 CRANIAL NERVES Motor CN V M L

35 CRANIAL NERVES TMJ CN V TMJ

36 CRANIAL NERVES Masseter CN V Masseter

37 CRANIAL NERVES Corneal CN V Corneal R

41

38 CRANIAL NERVES Corneal CN V Corneal L

39 CRANIAL NERVES VII Motor

40 CRANIAL NERVES VII SensAbsent

41 CRANIAL NERVES VIII Rub

42 CRANIAL NERVES VIII Webb

43 CRANIAL NERVES VIII Rinne

44 CRANIAL NERVES VIII Otoscope

45 CRANIAL NERVES IX/X Palate R

46 CRANIAL NERVES IX/X L Fatigue

47 CRANIAL NERVES IX/X Gag R

48 CRANIAL NERVES IX/X Gag L

49 CRANIAL NERVES IX/X Sens

50 CRANIAL NERVES XI XI

51 CRANIAL NERVES XI Trap

52 CRANIAL NERVES XII XII

53 CRANIAL NERVES XII Swallow

54 CRANIAL NERVES Blepharospasm

55 CRANIAL NERVES Anisocoria

56 CRANIAL NERVES Ptosis

57 CRANIAL NERVES Direct/Consensual

58 CRANIAL NERVES Lid Lag R L

59 CRANIAL NERVES Field of Gaze

60 CRANIAL NERVES Convergence

61 CRANIAL NERVES Tropia

62 CRANIAL NERVES Phoria

63 CRANIAL NERVES Nystagmus

64 CRANIAL NERVES OPK

65 CRANIAL NERVES VOR

66 CRANIAL NERVES Pursuit

67 CRANIAL NERVES Saccades

68 CRANIAL NERVES Hypermetria

69 CRANIAL NERVES Hypometria

70 CRANIAL NERVES Tracking

71 CBVS Toe Walk

72 CBVS Tandem Walk

73 CBVS Romberg

74 CBVS Pertubational

75 CBVS Fingerer to nose Fingerer to nose

76 CBVS Piano Playing Piano Playing

77 CBVS Rapid Alt hands Rapid Alt hands

42

78 CBVS Pyramidal Weak Pyramidal Weak 79 CBVS Dual Tasking

80 CBVS Heel Walk

81 CBVS Ellipse Direction

82 CBVS Fukuda test

83 CBVS Halmagi Head Thrust

84 CBVS Pull Test

85 CBVS Head Shake Nystagmus

86 CBVS Heel to Shin

87 CBVS Rapid Alt Feet Rapid Alt Feet

88 REFLEX PE MSR Jaw Jerk R

89 REFLEX PE MSR Jaw Jerk L

90 REFLEX PE MSR Triceps R

91 REFLEX PE MSR Triceps L

92 REFLEX PE MSR Biceps Br R

93 REFLEX PE MSR Biceps Br L

94 REFLEX PE MSR Brachloradialis R

95 REFLEX PE MSR Brachloradialis L

96 REFLEX PE MSR Patella R

97 REFLEX PE MSR Patella L

98 REFLEX PE MSR Achilles R

99 REFLEX PE MSR Achilles L

100 REFLEX PE MSR Hoffman R

101 REFLEX PE MSR Hoffman L

102 REFLEX PE MSR Plantar R

103 REFLEX PE MSR Plantar L

104 REFLEX PE Abdomen Auscult

105 REFLEX PE Abdomen Superficial Reflex T7-9

106 REFLEX PE Abdomen T10-11

107 SENSORY PE Pin Prick/Pinwheel Upper Extr Proximal R

108 SENSORY PE Pin Prick/Pinwheel Upper Extr Proximal L

109 SENSORY PE Pin Prick/Pinwheel Upper Extr Distal R

110 SENSORY PE Pin Prick/Pinwheel Upper Extr Distal L

111 SENSORY PE Pin Prick/Pinwheel Lower Ext Proximal R

112 SENSORY PE Pin Prick/Pinwheel Lower Ext Proximal L

113 SENSORY PE Pin Prick/Pinwheel Lower Ext Distal R

114 SENSORY PE Pin Prick/Pinwheel Lower Ext Distal L

115 SENSORY PE Pin Prick/Pinwheel Trunk Anterior R

116 SENSORY PE Pin Prick/Pinwheel Trunk Anterior L

117 SENSORY PE Pin Prick/Pinwheel Trunk Posterior R

43

118 SENSORY PE Pin Prick/Pinwheel Trunk Posterior L

119 SENSORY PE Vibration/Lt. Touch Upper Extr Proximal R

120 SENSORY PE Vibration/Lt. Touch Upper Extr Proximal L

121 SENSORY PE Vibration/Lt. Touch Upper Extr Distal R

122 SENSORY PE Vibration/Lt. Touch Upper Extr Distal L

123 SENSORY PE Vibration/Lt. Touch Lower Ext Proximal R

124 SENSORY PE Vibration/Lt. Touch Lower Ext Proximal L

125 SENSORY PE Vibration/Lt. Touch Lower Ext Distal R

126 SENSORY PE Vibration/Lt. Touch Lower Ext Distal L

127 SENSORY PE Vibration/Lt. Touch Lower Ext Distal R

128 SENSORY PE Vibration/Lt. Touch Lower Ext Distal L

129 SENSORY PE Joint Position Sense Upper Extr Distal R

130 SENSORY PE Joint Position Sense Upper Extr Distal L

131 SENSORY PE Joint Position Sense Lower Ext Proximal R

132 SENSORY PE Joint Position Sense Lower Ext Proximal L

133 SENSORY PE Joint Position Sense Lower Ext Distal R

134 SENSORY PE Joint Position Sense Lower Ext Distal L

135 SENSORY PE Additional Tests Stereognosis

136 SENSORY PE Additional Tests Graphesthesia

137 SENSORY PE Additional Tests Tactile Extinction

138 MOTOR Muscle Tone/Rigidity Deltoid R

139 MOTOR Muscle Tone/Rigidity Deltoid L

140 MOTOR Muscle Tone/Rigidity Bicep R

141 MOTOR Muscle Tone/Rigidity Bicep L

142 MOTOR Muscle Tone/Rigidity Tricep R

143 MOTOR Muscle Tone/Rigidity Tricep L

144 MOTOR Muscle Tone/Rigidity Wri Flx R

145 MOTOR Muscle Tone/Rigidity Wri Flx L

146 MOTOR Muscle Tone/Rigidity Wri Ext R

147 MOTOR Muscle Tone/Rigidity Wri Ext L

148 MOTOR Muscle Tone/Rigidity Finger Ext R

149 MOTOR Muscle Tone/Rigidity Finger Ext L

150 MOTOR Muscle Tone/Rigidity Finger Abd R

151 MOTOR Muscle Tone/Rigidity Finger Abd L

152 MOTOR Muscle Tone/Rigidity Hip Flex R

153 MOTOR Muscle Tone/Rigidity Hip Flex L

154 MOTOR Muscle Tone/Rigidity Knee Ext R

155 MOTOR Muscle Tone/Rigidity Knee Ext L

156 MOTOR Muscle Tone/Rigidity Ank Inver R

157 MOTOR Muscle Tone/Rigidity Ank Inver L

44

158 MOTOR Muscle Tone/Rigidity Ext Hal Long R 159 MOTOR Muscle Tone/Rigidity Ext Hal Long L

160 MOTOR Muscle Tone/Rigidity Ank Ever R

161 MOTOR Muscle Tone/Rigidity Ank Ever L

162 MOTOR Strength Deltoid R

163 MOTOR Strength Deltoid L

164 MOTOR Strength Bicep R

165 MOTOR Strength Bicep L

166 MOTOR Strength Tricep R

167 MOTOR Strength Tricep L

168 MOTOR Strength Wri Ext R

169 MOTOR Strength Wri Ext L

170 MOTOR Strength Wri Flex R

171 MOTOR Strength Wri Flex L

172 MOTOR Strength Finger Flex R

173 MOTOR Strength Finger Flex L

174 MOTOR Strength Finger Ext R

175 MOTOR Strength Finger Ext L

176 MOTOR Strength Finger Abduct R

177 MOTOR Strength Finger Abduct L

178 MOTOR Strength Finger Adduct R

179 MOTOR Strength Finger Adduct L

180 MOTOR Strength Exten Hal Long R

181 MOTOR Strength Exten Hal Long L

182 MOTOR Strength Dorsiflex R

183 MOTOR Strength Dorsiflex L

184 MOTOR Strength Plantarflex R

185 MOTOR Strength Plantarflex L

186 MOTOR Strength Invertors R

187 MOTOR Strength Invertors L

188 MOTOR Strength Evertors R

189 MOTOR Strength Evertors L

190 MOTOR Strength Hamstrings R

191 MOTOR Strength Hamstrings L

192 MOTOR Strength Quadriceps R

193 MOTOR Strength Quadriceps L

194 MOTOR Strength Hip Flex R

195 MOTOR Strength Hip Flex L

196 MOTOR Cervical ROM Flexion Normal

197 MOTOR Cervical ROM Flexion Pain

45

198 MOTOR Cervical ROM Extension Normal

199 MOTOR Cervical ROM Extension Pain

200 MOTOR Cervical ROM LLF Normal

201 MOTOR Cervical ROM LLF Pain

202 MOTOR Cervical ROM RLF Normal

203 MOTOR Cervical ROM RLF Pain

204 MOTOR Cervical ROM C1-C7 L Rot Normal

205 MOTOR Cervical ROM C1-C7 L Rot Pain

206 MOTOR Cervical ROM C1-C7 R Rot Normal

207 MOTOR Cervical ROM C1-C7 R Rot Pain

208 MOTOR Cervical ROM LFCC LFCC

209 MOTOR Cervical ROM MFCC MFCC

210 MOTOR D/L ROM Flexion Normal

211 MOTOR D/L ROM Flexion Pain

212 MOTOR D/L ROM Extension Normal

213 MOTOR D/L ROM Extension Pain

214 MOTOR D/L ROM LLF Normal

215 MOTOR D/L ROM LLF Pain

216 MOTOR D/L ROM RLF Normal

217 MOTOR D/L ROM RLF Pain

218 MOTOR D/L ROM L Rot Normal

219 MOTOR D/L ROM L Rot Pain

220 MOTOR D/L ROM R Rot Normal

221 MOTOR D/L ROM R Rot Pain

222 VITAL SIGNS HT

223 VITAL SIGNS WT

224 VITAL SIGNS TEMP

225 VITAL SIGNS BP Sitting R

226 VITAL SIGNS Standing R

227 VITAL SIGNS O2R

228 VITAL SIGNS PulseR

229 VITAL SIGNS Resp CPM

230 VITAL SIGNS Opththalmologic C/D

231 VITAL SIGNS Opththalmologic A/V

232 VITAL SIGNS Cardiovascular Heart Auscult

233 VITAL SIGNS Cardiovascular Valsalva

234 VITAL SIGNS Cardiovascular Color Facial

235 VITAL SIGNS Cardiovascular Capilliary refill

236 VITAL SIGNS Cardiovascular Color Hands

237 VITAL SIGNS Cardiovascular Apical/Radial Pulse

46

238 VITAL SIGNS Cardiovascular Radial Pulse R 239 VITAL SIGNS Cardiovascular Color Feet

240 VITAL SIGNS Cardiovascular Varicosities feet

241 VITAL SIGNS Cardiovascular Varicosities legs

242 VITAL SIGNS Cardiovascular Carotid Auscultation

243 VITAL SIGNS Cardiovascular Temporal Auscultation

244 VITAL SIGNS Cardiovascular Pedal Pulse R 0 1 2 3 4

245 VITAL SIGNS Cardiovascular Edema Edema Hands R

246 VITAL SIGNS Cardiovascular Edema Edema upper ext R

247 VITAL SIGNS Cardiovascular Edema Edema feet R

248 VITAL SIGNS Cardiovascular Edema Edema Lower Ext

249 VITAL SIGNS Respiratory Lung Auscultation

250 VITAL SIGNS Respiratory Percussion

251 VITAL SIGNS Respiratory Rib Expansion 0 1 2 3 4 cm

252 VITAL SIGNS Additional Tests Vetbas Artery Insuff

253 LIMBIC/META/OBS Abdomen Percussion

254 LIMBIC/META/OBS Abdomen Palpation

255 LIMBIC/META/OBS Abdomen Distention

256 LIMBIC/META/OBS Endocrine Palpation Thyroid

257 LIMBIC/META/OBS Endocrine Lymph Nodes

258 LIMBIC/META/OBS Endocrine Axillary

259 LIMBIC/META/OBS Skin Dry

260 LIMBIC/META/OBS Skin Cold

261 LIMBIC/META/OBS Skin Hot

262 LIMBIC/META/OBS Skin Sweaty

263 LIMBIC/META/OBS Skin Lesions UE

264 LIMBIC/META/OBS Skin Lesions LE

265 LIMBIC/META/OBS Skin Lesions Trunk

266 LIMBIC/META/OBS Skin Lesions Face

267 LIMBIC/META/OBS Skin Lesions Nails/Cuticles

268 BASAL GANGLIA Gait

269 BASAL GANGLIA Observations Spontaneous Movements

270 BASAL GANGLIA Observations Tics

271 BASAL GANGLIA Observations Vocalizations

272 BASAL GANGLIA Observations Tremors: Intention

273 BASAL GANGLIA Observations Tremors: Spontaneous

274 BASAL GANGLIA Observations Ballistic

275 BASAL GANGLIA Observations Choreiform

276 BASAL GANGLIA Observations Dyspraxia

277 BASAL GANGLIA Additional Tests Asterixes

47

278 BASAL GANGLIA Additional Tests Claudication

1 REVIEW & CLARIFICATION OF PRESCREEN HISTORY

2 DIAGNOSIS

3 DIFFERENTIAL DIAGNOSIS

4 SPECIAL STUDIES

5 TREATMENT/REHABILITATION

6 REFERRAL

7 ETHICS

48

Appendix H - Physical Examination - Item Analysis Statistics

Physical Examination – Item Analysis Statistics

Item# p-value

1 0.99

2 0.99

3 0.99

4 0.98

5 0.95

6 0.98

7 0.95

8 0.97

9 0.97

10 0.58

11 0.93

12 0.86

13 0.83

14 0.83

15 0.91

16 0.91

17 0.91

18 0.91

19 0.93

20 0.93

21 0.93

22 0.93

23 0.93

24 0.93

25 0.93

26 0.93

27 0.93

28 0.93

29 0.93

30 0.93

31 0.89

32 0.89

33 0.97

34 0.97

35 0.97

36 0.97

37 0.97

49


Item# p-value

38 0.97

39 0.97

40 0.91

41 0.96

42 0.97

43 0.95

44 0.94

45 0.97

46 0.83

47 0.95

48 0.90

49 0.86

50 0.97

51 0.91

52 0.96

53 0.87

54 0.92

55 0.95

56 0.96

57 0.95

58 0.87

59 0.94

60 0.94

61 0.96

62 0.95

63 0.93

64 0.97

65 0.87

66 0.97

67 0.92

68 0.84

69 0.85

70 0.91

71 0.99

72 0.97

73 1.00

74 0.89

75 0.99

76 0.96

50


Item# p-value

77 0.95

78 0.84

79 0.93

80 0.97

81 0.79

82 0.94

83 0.89

84 0.95

85 0.85

86 0.89

87 0.82

88 0.92

89 0.92

90 0.97

91 0.97

92 0.97

93 0.97

94 0.98

95 0.98

96 0.99

97 0.99

98 0.97

99 0.97

100 0.90

101 0.90

102 0.92

103 0.92

104 0.96

105 0.94

106 0.91

107 0.97

108 0.97

109 0.97

110 0.97

111 0.97

112 0.97

113 0.97

114 0.97

115 0.88

51


Item# p-value

116 0.88

117 0.88

118 0.88

119 0.93

120 0.93

121 0.95

122 0.95

123 0.95

124 0.95

125 0.97

126 0.97

127 0.97

128 0.97

129 0.95

130 0.95

131 0.93

132 0.93

133 0.95

134 0.95

135 0.97

136 0.99

137 0.85

138 0.96

139 0.96

140 0.97

141 0.97

142 0.97

143 0.97

144 0.96

145 0.96

146 0.96

147 0.96

148 0.96

149 0.96

150 0.96

151 0.96

152 0.96

153 0.96

154 0.97

52


Item# p-value

155 0.97

156 0.97

157 0.97

158 0.96

159 0.96

160 0.96

161 0.96

162 0.98

163 0.98

164 0.98

165 0.98

166 0.96

167 0.96

168 0.98

169 0.98

170 0.97

171 0.97

172 0.98

173 0.98

174 0.97

175 0.97

176 0.96

177 0.96

178 0.96

179 0.96

180 0.97

181 0.97

182 0.96

183 0.96

184 0.96

185 0.96

186 0.97

187 0.97

188 0.96

189 0.96

190 0.96

191 0.96

192 0.97

193 0.97

53


Item# p-value

194 0.97

195 0.97

196 0.96

197 0.72

198 0.96

199 0.72

200 0.95

201 0.71

202 0.95

203 0.71

204 0.97

205 0.71

206 0.96

207 0.71

208 0.81

209 0.82

210 0.93

211 0.70

212 0.93

213 0.69

214 0.93

215 0.70

216 0.93

217 0.70

218 0.92

219 0.71

220 0.92

221 0.71

222 1.00

223 1.00

224 1.00

225 1.00

226 1.00

227 0.99

228 0.99

229 1.00

230 0.97

231 0.97

232 1.00

54


Item# p-value

233 0.85

234 0.91

235 0.94

236 0.95

237 0.83

238 0.85

239 0.93

240 0.87

241 0.87

242 0.96

243 0.95

244 0.81

245 0.97

246 0.96

247 0.95

248 0.96

249 0.99

250 0.97

251 0.94

252 0.88

253 0.90

254 0.98

255 0.95

256 0.91

257 0.93

258 0.84

259 0.94

260 0.93

261 0.93

262 0.92

263 0.91

264 0.89

265 0.84

266 0.89

267 0.90

268 0.99

269 0.95

270 0.97

271 0.87

55


Item# p-value

272 0.91

273 0.91

274 0.90

275 0.89

276 0.89

277 0.86

278 0.89

56

Appendix I - Cases - Item Analysis Statistics

Cases – Item Analysis – p-values

Case

# N

Review & Clarification of

Prescreen History Diagnosis

Differential

Diagnosis

Special

Studies

Treatment

Rehabilitation Referral Ethics

1 12 1.00 1.00 .71 .92 1.00 .92 1.00

2 3 1.00 1.00 .50 1.00 .50 1.00 1.00

3 9 1.00 .89 1.00 .89 .89 .89 1.00

4 10 1.00 .90 .90 .90 .90 .90 1.00

5 17 1.00 .94 .91 .94 1.00 .88 .97

7 4 1.00 1.00 1.00 1.00 .75 1.00 1.00

9 5 1.00 .50 .50 .80 .60 .60 1.00

10 7 1.00 .57 .86 .71 1.00 1.00 1.00

11 8 1.00 1.00 .94 1.00 .94 1.00 1.00

57

Appendix J - Performance Examination - Frequency Distribution of Scores

Performance Examination – Frequency Distribution of Total Score

Percent Score Frequency Percent Cumulative Percent

37 1 1.3 1.3

66 1 1.3 2.7

72 1 1.3 4.0

74 1 1.3 5.3

75 1 1.3 6.7

77 1 1.3 8.0

79 1 1.3 9.3

81 1 1.3 10.7

81 1 1.3 12.0

83 1 1.3 13.3

84 1 1.3 14.7

85 1 1.3 16.0

86 1 1.3 17.3

87 1 1.3 18.7

88 1 1.3 20.0

88 1 1.3 21.3

89 1 1.3 22.7

89 1 1.3 24.0

89 1 1.3 25.3

89 1 1.3 26.7

90 1 1.3 28.0

90 1 1.3 29.3

91 1 1.3 30.7

92 1 1.3 32.0

92 1 1.3 33.3

92 1 1.3 34.7

92 1 1.3 36.0

93 1 1.3 37.3

94 1 1.3 38.7

94 1 1.3 40.0

95 2 2.7 42.7

95 1 1.3 44.0

95 1 1.3 45.3

96 1 1.3 46.7

96 1 1.3 48.0

96 1 1.3 49.3

96 1 1.3 50.7

96 1 1.3 52.0

97 1 1.3 53.3

97 2 2.7 56.0

97 1 1.3 57.3

97 1 1.3 58.7

97 2 2.7 61.3

98 3 4.0 65.3

98 1 1.3 66.7

58

Performance Examination – Frequency Distribution of Total Score

Percent Score Frequency Percent Cumulative Percent

98 1 1.3 68.0

98 2 2.7 70.7

99 1 1.3 72.0

99 2 2.7 74.7

99 1 1.3 76.0

99 1 1.3 77.3

99 1 1.3 78.7

99 4 5.3 84.0

99 1 1.3 85.3

99 1 1.3 86.7

99 2 2.7 89.3

100 2 2.7 92.0

100 3 4.0 96.0

100 2 2.7 98.7

100 1 1.3 100.0

Total 75 100

Documents

American Chiropractic Neurology Board, Inc. (ACNB) … · American Chiropractic Neurology Board, Inc. (ACNB) 2015 Diplomate Examination Technical Report April 10, ... Written Examination