Chapter 14Chapter 14

Measuring and Evaluating Measuring and Evaluating Knowledge Knowledge

and Assigning Gradesand Assigning Grades

Some Reasons to Test and GradeSome Reasons to Test and Grade

• To motivate students to study.To motivate students to study.

• To give students feedback on how To give students feedback on how much/how well they are learning.much/how well they are learning.

• To give instructors feedback on To give instructors feedback on effectiveness of instruction.effectiveness of instruction.

• To show competence in a profession To show competence in a profession and to become certified.and to become certified.

Orientation of a Course—Orientation of a Course—What It Will AccomplishWhat It Will Accomplish

• Content orientedContent oriented

• Assignment orientedAssignment oriented

• Student oriented:Student oriented:– Teachers need to consider who Teachers need to consider who

their students are and adapt the their students are and adapt the course to those students’ interests, course to those students’ interests, abilities, and personalities.abilities, and personalities.

Bloom’s Taxonomy of CognitionBloom’s Taxonomy of Cognition

Applications of Bloom’s TaxonomyApplications of Bloom’s Taxonomy

• Knowledge of fact:Knowledge of fact:– Simply learning isolated facts (i.e., the Simply learning isolated facts (i.e., the

length of a football field; lists of medical length of a football field; lists of medical terminology).terminology).

• Comprehension:Comprehension:– Understanding the nature and meaning of Understanding the nature and meaning of

facts (i.e., “Explain the relationship facts (i.e., “Explain the relationship between validity and reliability”).between validity and reliability”).

(continued)(continued)

Applications of Bloom’s TaxonomyApplications of Bloom’s Taxonomy(continued)(continued)

• Application:Application:– Applying one’s understanding in an Applying one’s understanding in an

appropriate situation (i.e., knowing the appropriate situation (i.e., knowing the symptoms of athletic injuries and using that symptoms of athletic injuries and using that knowledge to diagnose an injury).knowledge to diagnose an injury).

• Analysis:Analysis:– Taking something apart to study its Taking something apart to study its

components (i.e., a coach analyzes and components (i.e., a coach analyzes and studies an opponent’s offense and defense).studies an opponent’s offense and defense).

(continued)(continued)

Applications of Bloom’s TaxonomyApplications of Bloom’s Taxonomy(continued)(continued)

• Synthesis:Synthesis:– Putting something back together after taking it Putting something back together after taking it

apart (i.e., a coach synthesizes effective tactics apart (i.e., a coach synthesizes effective tactics and strategies).and strategies).

• Evaluation:Evaluation:– Making judgments about value (i.e., a coach Making judgments about value (i.e., a coach

selects players and makes decisions about selects players and makes decisions about offensive and defensive plays).offensive and defensive plays).

Types of Test QuestionsTypes of Test Questions

• Multiple choiceMultiple choice• MatchingMatching• TrueTrue–false–false• Fill-in-the-blankFill-in-the-blank• Short answerShort answer• Essay:Essay:

– The ideal test question to reach The ideal test question to reach the highest levels of Bloom’s the highest levels of Bloom’s taxonomy.taxonomy.

Your ViewpointYour Viewpoint

• As a student, do you prefer memorizing As a student, do you prefer memorizing facts and then repeating them in a facts and then repeating them in a multiple-choice test, or analyzing and multiple-choice test, or analyzing and synthesizing a problem in an essay synthesizing a problem in an essay test? Why? test? Why?

• Which is easier for you? Which do you Which is easier for you? Which do you think your instructors prefer?think your instructors prefer?

Test Questions Written at Various Test Questions Written at Various Levels of Bloom’s TaxonomyLevels of Bloom’s Taxonomy

Test QuestionsTest Questions(continued)(continued)

Evaluation Questions to Ask Evaluation Questions to Ask When Grading a TestWhen Grading a Test

• Have test takers misunderstood the questions?Have test takers misunderstood the questions?• Are students answering a question incorrectly Are students answering a question incorrectly

because they don’t understand it (rather than because they don’t understand it (rather than simply not knowing the answer)?simply not knowing the answer)?

• How can questions be improved?How can questions be improved?• Were there any questions that everyone, or most Were there any questions that everyone, or most

everyone, got correct/incorrect? Why?everyone, got correct/incorrect? Why?• Were there any questions to which the answers Were there any questions to which the answers

were especially weak? Why?were especially weak? Why?

Item Analysis Item Analysis for Evaluating Testsfor Evaluating Tests

• Calculating discriminationCalculating discrimination– Discrimination: The ability of a test to Discrimination: The ability of a test to

distinguish between those who have distinguish between those who have the relevant knowledge and those who the relevant knowledge and those who don’t.don’t.

• Calculating difficultyCalculating difficulty

Calculating DiscriminationCalculating Discrimination

• Divide the class into thirds based on their Divide the class into thirds based on their scores on a particular test.scores on a particular test. 42 42 ÷ 3 = 14÷ 3 = 14

• Subtract the number of those in the bottom third Subtract the number of those in the bottom third who got the question correct from those in the who got the question correct from those in the top third who got it correct.top third who got it correct. 6 in the top third and 2 in the bottom third got it right.6 in the top third and 2 in the bottom third got it right.

• Divide the answer by the number of students in Divide the answer by the number of students in each group.each group. 6 6 – 2 = 4– 2 = 4 4 ÷ 14 = 0.294 ÷ 14 = 0.29

Calculating DifficultyCalculating Difficulty

• Divide the number of students who Divide the number of students who answered the question correctly by answered the question correctly by the number who took the test.the number who took the test.

Number who took the test = 42Number who took the test = 42

Number of correct answers = 11Number of correct answers = 11

Difficulty index = 11 Difficulty index = 11 ÷ 42 = 0.26÷ 42 = 0.26

Standardized TestsStandardized Tests

• SAT, ACT, GRE, LSAT, etc.SAT, ACT, GRE, LSAT, etc.

• Primarily multiple-choice and truePrimarily multiple-choice and true–false –false questions.questions.

• Primarily norm-referenced Primarily norm-referenced

• Some controversy: Some controversy: – Cultural bias Cultural bias – Stressful nature of the testStressful nature of the test– Too much riding on test’s outcomeToo much riding on test’s outcome

Items to Consider When Grading Items to Consider When Grading Physical Activities and SportsPhysical Activities and Sports

• PerformancePerformance

• KnowledgeKnowledge

• FitnessFitness

• Psychomotor Psychomotor skillsskills

• Ability to think Ability to think and adjust quicklyand adjust quickly

• TeamworkTeamwork

• ParticipationParticipation

• ImprovementImprovement

• EffortEffort

Methods forMethods forCalculating GradesCalculating Grades

• Weighting scoresWeighting scores

• Using standard scores Using standard scores (grading on a curve)(grading on a curve)

Calculating a WeightedCalculating a WeightedGrade AverageGrade Average

Grading on a CurveGrading on a Curve

• z-scoresz-scores• A process of transforming scores; converting A process of transforming scores; converting

scores from one scale of numbers to another.scores from one scale of numbers to another.• A norm-referenced approach in which students A norm-referenced approach in which students

are compared to each other, not to an external are compared to each other, not to an external standard.standard.

• Students are forced into a competitive situation Students are forced into a competitive situation in which their grades depend on other students in which their grades depend on other students as well as their own knowledge.as well as their own knowledge.

An Example of aAn Example of aGrading PhilosophyGrading Philosophy

Errors in GradingErrors in Grading

• Too much emphasis on factual Too much emphasis on factual knowledgeknowledge

• Validity:Validity:– Invalid questionsInvalid questions– Invalid tests; test questions not related to Invalid tests; test questions not related to

a learning objectivea learning objective– Invalid scoringInvalid scoring

• Objectivity:Objectivity:– Subjectivity arises when someone decides Subjectivity arises when someone decides

what is the “best” answer.what is the “best” answer.

Tips for Grading Essay QuestionsTips for Grading Essay Questions

• Make up an answer key before grading Make up an answer key before grading and devise a grading scheme.and devise a grading scheme.

• Grade all answers to Question 1 before Grade all answers to Question 1 before moving on to grade Question 2.moving on to grade Question 2.

• Grade all answers to a particular question Grade all answers to a particular question at the same time.at the same time.

• Have an idea of what constitutes an A, a Have an idea of what constitutes an A, a B, and so on, before grading.B, and so on, before grading.– Use a rubric.Use a rubric.

Formative and Summative Formative and Summative Assessment in GradingAssessment in Grading

• Formative:Formative:– Diagnoses problems and evaluates progress.Diagnoses problems and evaluates progress.– Contributes to instruction by providing feedback.Contributes to instruction by providing feedback.

• Summative:Summative:– Compiles assessment data provided at the end of Compiles assessment data provided at the end of

unit or at other intervals.unit or at other intervals.– Assesses an individual’s degree of knowledge on a Assesses an individual’s degree of knowledge on a

particular subject.particular subject.– Describes what was learned or what skills were Describes what was learned or what skills were

acquired.acquired.

Your ViewpointYour Viewpoint

• Have you ever taken a poorly written Have you ever taken a poorly written test?test?

• Have you complained about invalid Have you complained about invalid questions or subjective grading to questions or subjective grading to your instructor? What was the result?your instructor? What was the result?