TWS AidTWS Aid

For Factor 6: Analysis of Assessment Procedures

MajorMajor Section Requirements:Section Requirements:

1. Graph comparing pre and post assessments2. Worksheet showing Learning Gain Scores

(LGS) for each student3. Learning Achievement Table depicting % of

students achieving “mastery level” for each objective

4. Text/Graph/Chart analyzing how subgroups performed relative to each objective

The Learning Achievement The Learning Achievement TableTablePurpose: To identify the percentage of students who achieved

a minimum mastery level for EACH learning objective.

All assessment data (tests, performances, portfolios, formal and informal questioning, checklists, rubrics, etc.) could be used in this table to calculate percentages.

Required in order to capture some forms of performance assessment that are criterion (given once at the end) assessments.

YOU set the criteria for what is considered mastery.

You are not going to be evaluated by degree of mastery.

Mastery Learning TableMastery Learning Table

Setting Mastery Levels

Example for: Knowledge Objective & Assessment TypeExample for: Knowledge Objective & Assessment TypeObjectiveObjective: List 12 reasons for the Industrial RevolutionAssessmentAssessment: Fill in the blank – paper and pencil testMastery LevelMastery Level: 9 out of 12 is considered mastery

Example for: a Performance Objective & Assessment TypeExample for: a Performance Objective & Assessment TypeObjectiveObjective: Student integrates technology throughout presentationAssessmentAssessment: Student presentations at the end of the unitMastery LevelMastery Level: Score of a 3 on a 1-3 rubric factor for technology (see

rubric)

Mastery should not be set too high or too low.Mastery should not be set too high or too low. You need to be able to examine each student’s You need to be able to examine each student’s

performance for each objective.performance for each objective.

Examples of Objectives Classified Low Level of Difficulty

Cognitive: Knowledge & Comprehension (Bloom); Knowledge (Sternberg)Affective: Receiving & Responding toPsychomotor: Reflex movements & Fundamental movements

In a given music example, the student should be able to identify all of the basic elements of music (cognitive)The student will report two likes and two dislikes as a response to the assigned reading (affective)The student performs a tennis serve with correct form (psychomotor)

Examples of ObjectivesClassified Middle Level of Difficulty

Cognitive: Application & Analysis (Bloom); Skills/Performance/Applications (Stenberg)

Affective: Valuing & organizing of valuesPsychomotor: Perceptual abilities & Physical abilities

After watching a video on scientific theories the student will list at least one difference between the use of the term “theory” in science and the use of “theory” in non-science contexts (cognitive)During a debate, the student will defend the right of scientists to conduct research in a three-minute statement (affective)Within the time allowed the student will adjust a microscope so that the image is clear (psychomotor)

Examples of ObjectivesClassified High Level of Difficulty

Cognitive: Synthesis & Evaluation (Bloom); Reasoning Ability (Stenberg)Affective: Internalizing valuesPsychomotor: Skilled movements & Nondiscursive communication

Given a map with six distinct geographical features, students will be able to evaluate the best location for building a new city (cognitive)Student members of jazz band will perform solo improvisations in Count Basie style for One O’Clock Jump (affective)The student accurately performs a tennis serve, including correct spin, speed and placement of the ball in the opposite side of the court (psychomotor)

Student 1 Student 2 Student 3 Student 4 Student 5 Student 6 Student 7

100%

80%

60%

40%

20%

0%

Graph for Pre/Post Assessment Data for Each Graph for Pre/Post Assessment Data for Each StudentStudent

Pre-Assessment Score

Post-Assessment Score

This example graph shows a comparison of the pre & post-assessment data for each student.

Concept of Gain Scores

A gain score is the actual gain divided by the potential gain.

GAIN SCORE = ACTUAL GAIN

POTENTIAL GAIN

.82

82%What does it mean?What does it mean?

Gain Score Worksheet

Pre Post GainStudent # Score Score Score

1 40 90 .832 90 95 .503 40 804 30 855 85 88 .206 75 100 +1.007 60 90 .758 40 95 .929 40 40 .0010 75 50 -1.0011 45 70 .4512 60 80 .5013 40 95 .92

SolveSolve

GAIN SCOREUsing the formula, Using the formula, calculate the calculate the missing gain scores.missing gain scores.

Gain Score Worksheet

Pre Post GainStudent # Score Score Score

1 40 90 .832 90 95 .503 40 80 .67.674 30 85 .79.795 85 88 .206 75 100 +1.007 60 90 .758 40 95 .929 40 40 .0010 75 50 -1.0011 45 70 .4512 60 80 .5013 40 95 .92

Include the average of the gain scores.

“ 50%” Average Learning Gain

GAIN SCORE

Solutions:80 – 40

100 – 404060

= .67=

85 – 30100 – 30

5570

.79==

Group Average .50

Negative Gain Scores

Example:Pre-Assessment = 75 %Post-Assessment = 50 %

Formula: Post-Assess % - Pre-Assess % 100% - Pre-Assess %

What happens when a student scores lower on the post-assessment than on the pre-assessment?

-25 25

50 - 75100 - 75

This student could have gained up to 25 points, but instead lost 25 This student could have gained up to 25 points, but instead lost 25 points, a value equaling 100 % of what could have been gained.points, a value equaling 100 % of what could have been gained.

= = = - 1.00-2525

Gain Scores for Pre-Assessments of Gain Scores for Pre-Assessments of 100%100%

What happens when a student scores 100% on the pre-assessment?

When a student scores 100% on the pre-assessment, that score must be changed to 99%. Otherwise, the potential gain will equal zero and the gain score will be undefined.

Changing the student’s pre-assessment score to 99% will make the potential gain equal 1. Then, using 1 as the denominator (instead of 0) will allow the score to be defined.

Actual Gain

0

Actual Gain Potential Gain

Gain Score == undefined

If the potential gain equals zero, then

Explanation:

Remember:

Other Points About Gain Scores

Gain scores are not a perfect measure

You should include more than just assessment of knowledge in your assessment plan and gain score calculations

You are not being evaluated or judged on the amount of gain you produce

Sub-Group ComparisonsSub-Group Comparisons

Purpose: To provide evidence that you can deal effectively with students with various needs and from various backgrounds.

YOU pick the groups of individuals

Could use SES, ESL/Non-ESL, race/ethnicity, disabilities, achievement levels, and gender as potential comparisons

Comparisons can be depicted: in text using a narrative while referring to gain scores and objective mastery % with a chart or graph

Example Graph for Sub-Group Comparison

100%

90%

80%

70%

60%

50%

40%

30%

20%

10%

0%

ESL,84%

Non-ESL,86%

Could also useGain Scores here.

Perc

en

tag

e