Reading Assessments for Elementary Schools
Tracey E. Hall
Center for Applied Special Technology
Marley W. WatkinsPennsylvania State University
Frank C. WorrellUniversity of California, Berkeley
REVIEW: Major Concepts
• Nomothetic and Idiographic• Samples• Norms• Standardized Administration• Reliability• Validity
Nomothethic
• Relating to the abstract, the universal, the general.
• Nomothetic assessment focuses on the group as a unit.
• Refers to finding principles that are applicable on a broad level.
• For example, boys report higher math self-concepts than girls; girls report more depressive symptoms than boys..
Idiographic
• Relating to the concrete, the individual, the unique
• Idiographic assessment focuses on the individual student
• What type of phonemic awareness skills does Joe possess?
Populations and Samples I
• A population consists of all the representatives of a particular domain that you are interested in
• The domain could be people, behavior, curriculum (e.g. reading, math, spelling, ...
Populations and Samples II
• A sample is a subgroup that you actually draw from the population of interest
• Ideally, you want your sample to represent your population– people polled or examined, test content, manifestations of behavior
Samples
• A random sample is one in which each member of the population had an equal and independent chance of being selected.
• Random samples are important because the idea is to have a sample that represents the population fairly; an unbiased sample.
• A sample can be used to represent the population.
• Sampling in which elements are drawn according to some known probability structure.
• Probability samples are typically used in conjunction with subgroups (e.g., ethnicity, socioeconomic status, gender).
Norms I
• Norms are examples of how the “average” individual performs.
• Many of the tests and rating scales that are used to compare children in the US are norm-referenced.– An individual child’s performance is compared to the norms established using a representative sample.
Norms II
• For the score on a normed instrument to be valid, the person being assessed must belong to the population for which the test was normed
• If we wish to apply the test to another group of people, we need to establish norms for the new group
Norms III
• To create new norms, we need to do a number of things:– Get a representative sample of new population
– Administer the instrument to the sample in a standardized fashion.
– Examine the reliability and validity of the instrument with that new sample
– Determine how we are going to report on scores and create the appropriate tables
Standardized Administration
• All measurement has error.• Standardized administration is one way to reduce error due to examiner/clinician effects.
• For example, consider these questions with different facial expressions and tone:
• Please define a noun for me :-)• DEFINE a noun if you can ? :- (
Normal Curve
• Many distributions of human traits form a normal curve
• Most cases cluster near middle, with fewer individuals at extremes; symmetrical
• We know how the population is distributed based on the normal curve
Ways of Reporting Scores
• Mean, standard deviation• Distribution of scores
– 68.26% ± 1; 95.44 ± 2; 99.72 ±3• Stanines (1, 2, 3, 4, 5, 6, 7, 8, 9)
• Standard scores - linear transformations of scores, but easier to interpret
• Percentile ranks* • Box and Whisker Plots*
Percentiles
• A way of reporting where a person falls on a distribution.
• The percentile rank of a score tells you how many people obtained a score equal to or lower than that score.
• Box and whisker plots are visual displays or graphic representations of the shape of a distribution using percentiles.
The box plot is a picture of the distribution of scores on a measure.
Explanation of the
Box Plot
Individual
Outliers
90th Percentile
Performance
75th Percentile
50th
Percentile
25th Percentile
10th Percentile
0
2
4
6
8
10
12
14
16
18
20
Grade 2 Students
Correlation
• We need to understand the correlation coefficient to understand the manual
• The correlation coefficient, r, quantifies the relationship between two sets of scores
• A correlation coefficient can have a range from -1 to + 1– Zero means the two sets of scores are not related.
– One means the two sets of scores are identical (a perfect correlation)
Correlation 2
• Correlations can be positive or negative.• A + correlation tells us that as one set of scores increases, the second set of scores also increases. they can be negative. Examples?
• A negative correlation tells us that as one set of scores increases, the other set decreases. Think of some examples of variables with negative r’s.
• The absolute value of a correlation indicates the strength of the relationship. Thus .55 is equal in strength to -.55.
How would you describe the correlations shown
by these charts?3 15 47 39 7
11 913 10
1098765
1.21.21.21.21.2
0
2
4
6
8
10
12
14
1 2 3 4 5 6
Series1
0
2
4
6
8
10
12
1 2 3 4 5 6
Series1
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 2 3 4 5
Series1
Reliability
• Reliability addresses the stability, consistency, or reproducibility of scores.– Internal consistency
– Split half, Cronbach’s alpha
– Test-retest– Parallel/Alternate forms– Inter-rater
Validity
• Validity addresses the accuracy or truthfulness of scores. Are they measuring what we want them to?– Content– Criterion - Concurrent– Criterion - Predictive– Construct
– Face– (Cash)
Content Validity
• Is the assessment tool representative of the domain (behavior, curriculum) being measured?
• An assessment tool is scrutinized for its (a) completeness or representativeness, (b) appropriateness, (c) format, and (d) bias– E.g., MSPAS
Criterion-related Validity
• What is the correlation between our instrument, scale, or test and another variable that measures the same thing, or measures something that is very close to ours?
• In concurrent validity, we compare scores on the instrument we are validating to scores on another variable that are obtained at the same time.
• In predictive validity, we compare scores on the instrument we are validating to scores on another variable that are obtained at some future time.
Construct Validity
• Overarching construct: Is the instrument measuring what it is supposed to?– Dependent on reliability, content and criterion-related validity.
• We also look at some other types of validity some times– Convergent validity: r with similar construct
– Discriminant validity: r with unrelated construct
– Structural validity: What is the structure of the scores on this instrument?
Elementary Normative Sample
• Stratified by educational region
• Males and females represented equally.
• School, class, and individuals chosen at random.
• Final sample consists of 700 students (50% female).
Table 1.1
Normative Sample by Educational Region
Population Norm Sample
Division Number Percent Number PercentSt. George West 31,948 24.1 160 22.9St. George East 14,255 16.5 116 16.6St. Andrew/St. David 3,859 6.3 44 6.3Caroni 10,913 15.0 105 15.0Nariva/M ayaro 2,287 3.7 29 4.1Victoria 26,197 19.0 133 19.0St. Patrick 12,711 11.0 80 11.4Tobago 3,059 4.3 33 4.7Total 105,229 100.0 700 100.0
p. 2
Table 1.2
Normative Sample by Age
Age Number Percent
4 18 2.6 5 88 12.6 6 96 13.7 7 98 14.0 8 98 14.0 9 99 14.1 10 95 13.6 11 89 12.7 12 14 2.0 13 4 0.6 14 1 0.1 p. 2
Table 1.3
Normative Sample by Gender
Gender Number Percent
Male 350 50.0Female 350 50.0
p. 2
Table 1.4
Normative Sample by Grade
Grade Number Percent
1st Year 100 14.32nd Year 100 14.3Standard 1 100 14.3Standard 2 100 14.3Standard 3 100 14.3Standard 4 100 14.3Standard 5 100 14.3
p. 3
Table 1.5
Normative Sample by Ethnic Background
Ethnic Background Number Percent
African 269 38.4East Indian 257 36.7M ixed 149 21.3Other 1 0.1Not Reported 24 3.4
p. 3
Table 1.6
Normative Sample by Parental Education Level
Mothers Fathers
Highest level completed Number Percent Number Percent
Primary 209 29.9 216 30.9Form 3 49 7.0 31 4.4Form 5 203 29.0 169 24.1Form 6 15 2.1 16 2.3University 5 0.7 10 1.4Unknown 219 31.3 258 36.9
p. 3
Measures
• First and Second Year/Infants 1 and 2– Mountain Shadows Phonemic Awareness Scale (MS-PAS) - group administered.
– Individual Phonemic Analysis
• Second Year/Infant 2 to Standard 5– Oral Reading Fluency
• Standards 1 and 2– The Cloze Procedure
Assessment• Determine starting point • Analyze Errors• Monitor Progress• Modify Instruction
Instructional Delivery• Secure student attention• Pace instruction appropriately• Monitor student performance• Provide feedback
Instructional Design• Determine Content• Select Language of Instruction• Select examples• Schedule scope and sequence• Provide for cumulative review
Initial Evaluation• Archival Assessment• Diagnostic Assessments• Formal Standardized Measures
Madigan, Hall, & Glang(1997)
Assessment Instruction Cycle