Upload
eric-armstrong
View
233
Download
0
Tags:
Embed Size (px)
Citation preview
Completion, Completion, Short-Answer, and Short-Answer, and True-False ItemsTrue-False Items
Educational Assessment of Educational Assessment of StudentsStudentsChapter 7Chapter 7
Anthony J. NitkoAnthony J. Nitko(Brenda Stewart)(Brenda Stewart)
Three Fundamental Three Fundamental Principles for Crafting Principles for Crafting
AssessmentsAssessments Focus each assessment task entirely on Focus each assessment task entirely on
important learning targetsimportant learning targets Craft each assessment task to elicit from Craft each assessment task to elicit from
students only the knowledge and students only the knowledge and performance that are relevant to the performance that are relevant to the learning targets you are assessinglearning targets you are assessing
Craft each assessment task to neither Craft each assessment task to neither prevent nor inhibit a student’s ability to prevent nor inhibit a student’s ability to demonstrate attainment of the leaning demonstrate attainment of the leaning targets you are assessingtargets you are assessing
Short-Answer ItemsShort-Answer Items
Require a word, short phrase, number, Require a word, short phrase, number, or symbol response.or symbol response.
Three types of short-answer items:Three types of short-answer items: Question – What is the capital of Texas?Question – What is the capital of Texas? Completion – The capital of Texas is Completion – The capital of Texas is
_________._________. Association – On the blank next to the state Association – On the blank next to the state
name, write the abbreviation for that state.name, write the abbreviation for that state. TexasTexas TXTX CaliforniaCalifornia CACA
Usefulness of Short-Usefulness of Short-Answer ItemsAnswer Items
Abilities AssessedAbilities Assessed Lower-order thinking skillsLower-order thinking skills
RecallRecall Comprehension of informationComprehension of information
Higher-level abilitiesHigher-level abilities Make simple interpretations of data and Make simple interpretations of data and
applications of rulesapplications of rules Ability to solve numerical problems in science Ability to solve numerical problems in science
and mathematicsand mathematics Ability to manipulate mathematical symbols Ability to manipulate mathematical symbols
and balance mathematical and chemical and balance mathematical and chemical equationsequations
Usefulness of Short-Usefulness of Short-Answer ItemsAnswer Items
StrengthsStrengths Relatively easy to constructRelatively easy to construct Can be scored objectivelyCan be scored objectively Lowers the probability of getting the Lowers the probability of getting the
answer correct by random guessinganswer correct by random guessing ShortcomingsShortcomings
Not free of subjectivityNot free of subjectivity Cannot anticipate all possible responsesCannot anticipate all possible responses Tends to lower the reliability of the Tends to lower the reliability of the
obtained scoreobtained score
Crafting Short-Answer Crafting Short-Answer ItemsItems
Importance of what is assessedImportance of what is assessed How does it match the test blueprintHow does it match the test blueprint Question format is the preferred format Question format is the preferred format
for a short-answer item. Focus the item on for a short-answer item. Focus the item on the specific knowledge soughtthe specific knowledge sought
Word the items specifically and clearlyWord the items specifically and clearly Put the blank near the end of the sentencePut the blank near the end of the sentence If possible do not copy statements If possible do not copy statements
verbatimverbatim
Crafting Short-Answer Crafting Short-Answer ItemsItems
A completion item should omit A completion item should omit important words and not trivial important words and not trivial wordswords
Limit blanks to one or two Limit blanks to one or two Keep all blanks the same lengthKeep all blanks the same length Specify the precision you expect in Specify the precision you expect in
your answeryour answer Avoid irrelevant cluesAvoid irrelevant clues
True-False ItemsTrue-False Items
Consists of a statement or proposition Consists of a statement or proposition the student may judge as true or falsethe student may judge as true or false
Six varieties:Six varieties: True-FalseTrue-False Yes-NoYes-No Right-WrongRight-Wrong CorrectionCorrection Multiple True-FalseMultiple True-False Yes-No with explanationYes-No with explanation
Usefulness of True-False Usefulness of True-False ItemsItems
AdvantagesAdvantages Certain aspects of the subject matter readily lend Certain aspects of the subject matter readily lend
themselves to verbal propositions that can be themselves to verbal propositions that can be judged true or falsejudged true or false
Relatively easy to writeRelatively easy to write Scored easily and objectivelyScored easily and objectively Can cover a wide range of content within a Can cover a wide range of content within a
relatively short periodrelatively short period CriticismsCriticisms
Poorly constructed assess only specific, frequently Poorly constructed assess only specific, frequently trivial factstrivial facts
Ambiguously wordedAmbiguously worded Answered by random guessingAnswered by random guessing Encourage students to study and accept only Encourage students to study and accept only
oversimplified statements of truth and factual oversimplified statements of truth and factual detailsdetails
Usefulness of True-False Usefulness of True-False ItemsItems
Generalizations in a subject areaGeneralizations in a subject area Comparisons among conceptsComparisons among concepts Casual or conditional propositionsCasual or conditional propositions Relationships between two events, Relationships between two events,
concepts, facts, or principlesconcepts, facts, or principles Explanations for why events or Explanations for why events or
phenomena occurredphenomena occurred Instances or examples of a concept or Instances or examples of a concept or
principleprinciple
Usefulness of True-False Usefulness of True-False ItemsItems
Evidential statementsEvidential statements Predictions about phenomena or Predictions about phenomena or
eventsevents Steps in a procedure of processSteps in a procedure of process Computations (or other kinds of Computations (or other kinds of
results obtained from applying a results obtained from applying a procedure)procedure)
Evaluations of events of phenomenaEvaluations of events of phenomena
Validity of True-False Item Validity of True-False Item FormatFormat
Essence of educational achievement is Essence of educational achievement is the command of useful verbal knowledgethe command of useful verbal knowledge
All verbal knowledge can be expressed in All verbal knowledge can be expressed in propositionspropositions
Proposition is any sentence that can be Proposition is any sentence that can be said to be true or falsesaid to be true or false
Particular area of knowledge is indicated Particular area of knowledge is indicated by success in judging truth or falsity of by success in judging truth or falsity of propositionspropositions
Guessing on True-False Guessing on True-False Items:Items:
An Ill Wind?An Ill Wind? Guessing – getting 50% correctGuessing – getting 50% correct
Blind (complete random guessing) quite Blind (complete random guessing) quite unlike informed guessing (based on partial unlike informed guessing (based on partial knowledge)knowledge)
Well-motivated guess blindly only on a Well-motivated guess blindly only on a small percentage of questions on a testsmall percentage of questions on a test
Difficult to obtain a good score on a test by Difficult to obtain a good score on a test by blind guessing onlyblind guessing only
A given true-false test has a high reliability A given true-false test has a high reliability coefficient – evidence blind guessing does coefficient – evidence blind guessing does not seriously affect the scoresnot seriously affect the scores
Guessing on True-False Guessing on True-False Items:Items:
An Ill Wind?An Ill Wind? Random guessing – sometimes called Random guessing – sometimes called
blind guessingblind guessing Partial knowledge – make an informed Partial knowledge – make an informed
guessguess Higher than 50-50 chance of successHigher than 50-50 chance of success Measures of partial knowledgeMeasures of partial knowledge
Full knowledge – 100% chance of Full knowledge – 100% chance of answering correctlyanswering correctly
Percentages of correct answers go down Percentages of correct answers go down with the number of blind guesses madewith the number of blind guesses made
Suggestions for Getting Suggestions for Getting Started ProperlyStarted Properly
Create pairs of items, one true and Create pairs of items, one true and one falseone false
Rephrase evaluative judgments into a Rephrase evaluative judgments into a comparative statementcomparative statement
Write false statements that reflect the Write false statements that reflect the actual misconceptions held by actual misconceptions held by studentsstudents
Convert a multiple-choice item into Convert a multiple-choice item into two or more true-false itemstwo or more true-false items
Suggestions for ImprovingSuggestions for ImprovingTrue-False ItemsTrue-False Items
Importance of what is assessedImportance of what is assessed How does it match the test blueprintHow does it match the test blueprint Assess important ideas, rather than Assess important ideas, rather than
trivia, general knowledge, or common trivia, general knowledge, or common sensesense
Make sure it is either definitely true or Make sure it is either definitely true or definitely falsedefinitely false Use short statements when possibleUse short statements when possible Use exact languageUse exact language Use positive statements and avoid double Use positive statements and avoid double
negativesnegatives
Suggestions for ImprovingSuggestions for ImprovingTrue-False ItemsTrue-False Items
Avoid copying sentence verbatimAvoid copying sentence verbatim True and false statements should have True and false statements should have
approximately the same number of wordsapproximately the same number of words Don’t present in a repetitive or easily Don’t present in a repetitive or easily
learned patternlearned pattern Do not use verbal clues that give away the Do not use verbal clues that give away the
answeranswer Attribute the opinion in a statement to an Attribute the opinion in a statement to an
appropriate sourceappropriate source Focus on one ideaFocus on one idea
Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items
Looks like a multiple-choice item – Looks like a multiple-choice item – followed by several alternativesfollowed by several alternatives
No single correct or best answerNo single correct or best answer Response to each alternative is true Response to each alternative is true
or falseor false Each alternative is scored correct or Each alternative is scored correct or
incorrectincorrect
Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items
AdvantagesAdvantages Two or three multiple true-false Two or three multiple true-false
responses in the time it takes to make responses in the time it takes to make one multiple-choice responseone multiple-choice response
Multiple true-false tests created from Multiple true-false tests created from multiple-choice items have higher multiple-choice items have higher reliability than original multiple-choicereliability than original multiple-choice
Can assess same abilities as straight Can assess same abilities as straight multiple-choice items that are crafted to multiple-choice items that are crafted to assess parallel contentassess parallel content
Crafting Multiple True-Crafting Multiple True-False ItemsFalse Items
AdvantagesAdvantages Students believe multiple true-false items Students believe multiple true-false items
do a better job of assessing knowledge do a better job of assessing knowledge than multiple-choice itemsthan multiple-choice items
Students perceive them to be harderStudents perceive them to be harder May be easier to write – not limited to one May be easier to write – not limited to one
correct answercorrect answer LimitationsLimitations
Shares some of same limitations as Shares some of same limitations as multiple-choicemultiple-choice
Works CitedWorks Cited
Nitko, A. J., (2004). Educational Nitko, A. J., (2004). Educational Assessment of Assessment of Students, 4Students, 4thth ed. ed. Upper Saddle River, NJ: Upper Saddle River, NJ: Pearson Pearson Education, Inc.Education, Inc.