measuring complex achiemvents

7/29/2019 measuring complex achiemvents

1/25

Measuring Complex

Achievement: Essay

Questions


2/25

Objectives for Chapter 10

Discuss the use of essay questions

Compare two types of essay questions

Hypothesize the advantages anddisadvantages of using essay questions

Compare two types of scoring rubrics

Evaluate sample essays using a scoring

rubricCreate examples of two types of essay

questions


3/25

Whats So Great About Essay

Questions?They allow assessment of students

abilities to:

Formulate problemsOrganize, integrate, and evaluate ideas

and information

Apply knowledge and skills


4/25

Forms and Uses of Essay

Questions

May be used to measure knowledge of

factual information Full potential realized with more higher-

order thinking problems


5/25

Useful in meeting learning outcomes

concerned with the abilities to

conceptualize, construct, organize,

integrate, relate, and evaluate ideas in

content areas such as: history, civics,

literature, science, math, etc.Most widely used when the main focus is

on student writing in any subject

Examples: distinguishing between

narrative essays, expository essays, andpersuasive essays or focusing on writing

process


6/25

Confucius Say

The freedom of response provided by essayquestions is not an all-or-nothing affair but,

rather, a matter of degrees.

RestrictedResponse

Extended-Response


7/25

Restricted Response Essay

QuestionsUsually limits both the content and the

response by restricting the scope of thetopic to be discussed, generallyindicated in the question

Useful for measuring learning outcomesrequiring interpretation and application ofdata in a specificarea

Any outcomes measured by an objectiveinterpretive exercise can be measuredby a restricted response essay question


8/25

Examples of Restricted

Response Essay QuestionsDescribe two situations that demonstrate

the application of the law of supply anddemand. Do not use those examplesdiscussed in class.

State the main differences between theVietnam War and previous wars in whichthe United States has participated.

Why is the barometer one of the mostuseful instruments for forecastingweather? Answer in a brief paragraph.


9/25

The Restricted Response essay questionprovides for more ease of assessment,but it restricts the scope of the topic to

be discussed and indicates the nature ofthe desired response to the studentwhich limits his or her opportunity to

demonstrate these behaviors on theirown.

Is there a better method for evaluatingcomplex achievement


10/25

Extended-Response Essay

Questions Freedom of response allows student to

select information that they think is

pertinent, to organize the answer inaccordance with their best judgment,

and to integrate and evaluate ideas as

they deem appropriate

Places value on higher-order thinkingskills


11/25

Examples of Extended-

Response Essay Questions Imagine that you and a friend found a

magic wand. Write a story about an

adventure that you and your friend hadwith the magic wand.

Compare developments in international

relations in the administrations of

President William Clinton and PresidentGeorge W. Bush. Cite examples when

possible.


12/25

To Keep Things Simple

Objective Interpretive- select

Restricted Response Essay- supply

Extended-Response Essay- write


13/25

Pros & Cons of Essay

Questions Apply your knowledge

They give student the

options to chose what

they want to write about reasoning

The teacher can tell if the

student studies or not

Can often prepare in

advance what to write

Time consuming

Grading: grammar usage

vs. content

Legibility

Hard to formulate ideas

in time

Takes longer to grade


14/25

How Can We Change Those

Cons to Pros?Unreliability- clearly defining learning

outcomes to be measured, properly

framing questions, carefully followingscoring rules, and obtaining practice in

scoring

Amount of time- reserve use of

extended-response questions forlearning outcomes that cannot be

measured well objectively


15/25

Limited Sampling- try to obtain as

representative a sample of learning

outcomes as possible (accumulate a

series of essays throughout the schoolyear to be included in a writing portfolio)


16/25

Suggestions for Constructing

Essay QuestionsRestrict use of essays to learning

outcomes that cannot be measured well

objectivelyConstruct questions that call forth skills

specified in learning standards

Example box on page 235


17/25

Phrase the question so that the students

task is clearly indicated

Example:

Poor: Compare the Democratic andRepublican parties.


18/25

Better: Compare the current policies of

the Democratic and Republican partieswith regard to the role of government in

private business. Support your

statements with examples when

possible. (Your answer should be

confined to two pages. It will be

evaluated in terms of the

appropriateness of the facts andexamples presented and the skill with

which it is organized.)


19/25

Indicate an approximate time limit for

each question

Avoid the use of optional questions Example Box page 239

Now that we have essay questions how dowe score them?


20/25

Scoring Essay Questions

Tips to remember

Use clear specifications of scoring

criteria Inform students of scoring criteria

Use an initial review to find anchorresponses for comparison

Use descriptive rather than judgmentalscores or levels (writing is clear andthoughts are complete vs. excellent)


21/25

Scoring for Restricted

Response Essay Questions In most instances, the teacher should

write an example of an expected

response For example, if the student is asked to

describe three factors that contributed to

the start of the Civil War, the teacher would

construct a list of acceptable reasons andgive the student 1 point for each of up to

three reasons given from the list


22/25

Scoring for Extended-

Response Essay QuestionsAnalytic Scoring Rubrics

Consist of a rubric broken down into key

dimensions that will be evaluated Enables teacher to focus on one

characteristic of a response at a time

Provides maximum feedback for

students


23/25

Holistic Scoring Rubrics

Yield a single overall score taking into

account the entire responseCan be used to grade essays more

quickly

Does not provide as much specificfeedback as analytic rubric

Should not consist of scores alone, butrather contain scores accompanied bystatements of the characteristics of theresponse

Example Table 10.3 and 10.4


24/25

Suggestions for Scoring Essay

Questions Prepare an outline of the expected

answer in advance and use a clear

scoring rubricUse the scoring rubric that is most

appropriate

Decide how to handle factors that are

irrelevant to the learning outcomes beingmeasured


25/25

Evaluate all responses to one question

before going on to the next one

When possible, evaluate answers

without looking at the students name

If especially important decisions are to

be based on the results, obtain two ormore independent ratings

Look out for bluffing! Page 247

Documents

measuring complex achiemvents