Developing Quality Open Response and Multiple Choice Items for the Classroom Kentucky Department of Education

Developing Quality Developing Quality Open Response and Open Response and

Multiple Choice Items Multiple Choice Items for the Classroomfor the Classroom

Kentucky Department of EducationKentucky Department of Education

PresentersPresenters

Sean Elkins and Mark KoppSean Elkins and Mark KoppCurriculum ConsultantsCurriculum ConsultantsKentucky Department of EducationKentucky Department of EducationOffice of Teaching and LearningOffice of Teaching and Learning500 Mero St./18th Floor500 Mero St./18th FloorFrankfort, KY 40601Frankfort, KY 40601(502) 564-2106 (502) 564-2106 Fax: (502) 564-9848Fax: (502) [email protected]@education.ky.gov [email protected]@education.ky.gov

IntroductionIntroduction

The purpose of the manual is to help you The purpose of the manual is to help you develop and incorporate KCCT-like questions develop and incorporate KCCT-like questions into your classroom instruction into your classroom instruction This manual is available as a downloadable file This manual is available as a downloadable file (PDF). It can be accessed from the Teaching (PDF). It can be accessed from the Teaching Tools page of the KDE website, or by clicking on Tools page of the KDE website, or by clicking on the following link:the following link:

http://www.education.ky.gov/KDE/Instructional+Rehttp://www.education.ky.gov/KDE/Instructional+Resources/Curriculum+Documents+and+Resources/sources/Curriculum+Documents+and+Resources/Teaching+Tools/Developing+Quality+Open+RespTeaching+Tools/Developing+Quality+Open+Response+and+Multiple+Choice+Items+for+the+Classronse+and+Multiple+Choice+Items+for+the+Classroom.htmoom.htm

Limitations of State AssessmentsLimitations of State Assessments

The Kentucky Core Content Test (KCCT) is:The Kentucky Core Content Test (KCCT) is:– Limited to Core Content for Assessment Limited to Core Content for Assessment

StandardsStandards– Limited to Depth of Knowledge (DOK) ceilingsLimited to Depth of Knowledge (DOK) ceilings– Limited to Multiple Choice (MC) and Open Limited to Multiple Choice (MC) and Open

Response (OR) questions (with the exception Response (OR) questions (with the exception of the Writing Assessments)of the Writing Assessments)

The Manual—An OverviewThe Manual—An Overview

The manual consists of 6 sections:The manual consists of 6 sections:1.1. Planning an AssessmentPlanning an Assessment

2.2. Writing an Open Response QuestionWriting an Open Response Question

3.3. The Five Basic Open Response Question The Five Basic Open Response Question TypesTypes

4.4. Developing a Scoring GuideDeveloping a Scoring Guide

5.5. Classroom Practices for Improving Student Classroom Practices for Improving Student ResponsesResponses

6.6. Writing a Multiple Choice QuestionWriting a Multiple Choice Question

Planning the AssessmentPlanning the Assessment

Open response and multiple choice questions should be included within the unit of study as assessments of the content and skills of that unit, not as isolated activitiesDeveloping quality assessments is an important step in constructing an instructional unit


All questions should be authentic and directly related to the standards (e.g., Academic Expectations, Program of Studies, and Core Content for Assessment) addressed in the unit of study


While both OR and MC items can potentially be used to assess any standard, the decision as to which is most appropriate depends upon the specific learning target you wish to address


Most content standards need to be ‘deconstructed’ into learning targets that specify what student learning and skill/ability should be demonstrated


Core Content StandardPL-05-1.1.2Students will recommend effective strategies for responding to stress, conflict, peer pressure and bullying (e.g., fairness, compromise, standing up for one’s rights, anger management, problem-solving, refusal skills, verbal/nonverbal communication). DOK 2

Student Learning Target

Students can describe and demonstrate a variety of ways to refuse drugs or alcohol

What is an Open Response question?What is an Open Response question?

Requires students to both demonstrate Requires students to both demonstrate content knowledge and to apply that content knowledge and to apply that knowledge in some wayknowledge in some way

It is the application component of an open It is the application component of an open response question that distinguishes it response question that distinguishes it from the more familiar essay or from the more familiar essay or constructed response questionconstructed response question

Advantages of Open Response ItemsAdvantages of Open Response Items

OR items allow for more depth of knowledge to be demonstrated than do MC items.

OR items allow students to demonstrate complex cognitive behaviors, such as comparing, relating, analyzing, inferring, concluding, predicting, generalizing, solving and/or applying.

Disadvantages of Open Response ItemsDisadvantages of Open Response Items

OR items are more difficult and more time consuming to scoreBecause of the time required to answer them, there must usually be fewer open response items on an assessment than MC itemsEffectiveness of OR items is dependant on the creation and use of a quality scoring guide and answer information provided

What is a Multiple Choice Question?What is a Multiple Choice Question?

‘‘Selected response’ itemsSelected response’ items

Begins with an item stem, followed by response choices

Can be used to measure both recall and higher order thinking

Advantages of Multiple Choice ItemsAdvantages of Multiple Choice Items

Can be used to measure a wide variety of learning outcomes

Permit wide sampling and broad coverage of a content domain

Are reliable and efficient to score

Can provide useful diagnostic information about the learning of individual students or groups of students

Disadvantages of Multiple Choice ItemsDisadvantages of Multiple Choice Items

MC items are difficult to write wellMC items cannot measure certain types of skills (e.g., the ability to organize and express ideas in writing; conduct a scientific investigation)MC item performance can be influenced by student characteristics unrelated to the content, such as reading ability and “test-wiseness”

Writing an Open Response QuestionWriting an Open Response Question

Items developed for classroom use are not limited to addressing Core Content for Assessment standards

Teachers are free to exceed the DOK ceilings for any Core Content for Assessment standard when developing items

Characteristics of a KCCT-like Characteristics of a KCCT-like Open ResponseOpen Response

Directly tied to one or more content standards

Consists of an item name, prompt and directions (question)

Specifies exactly what a student is required to do in order to achieve the maximum score. No extension is required beyond what the question specifies.


Requires a student to:– demonstrate content knowledge;– apply that knowledge; and– communicate an answer in no more than a

one-page written response

Scored by the use of a question-specific rubric on a 0-4 scale, but a general scoring guide is available to guide student responses


Written in one of five basic question formatsOR questions may:– have a correct answer which students can determine

and explain through a variety of methods or in varying degrees of correctness;

– have multiple successful answers for which students must apply their analytical skills to a response;

or– combine requirements: one part requires a student to

provide a single correct answer and a subsequent part asks the student to extend his/her knowledge in another way, such as applying the knowledge to another situation or by predicting an outcome


Native American Influences

Native American cultures have influenced many parts of American life, including our government. For example, in the Iroquois culture, each tribe would send a representative to meet with the other tribes’ representatives to discuss problems and make decisions for the entire Iroquois League.

a. Explain how our government uses this idea of representative government.b. Explain TWO reasons why this is a good way to govern our country.

Item namePrompt

Directions

The Five Basic The Five Basic Open Response Question TypesOpen Response Question Types

1. Scaffolded questions2. Single Dimension/Component

questions3. Two or More Relatively Independent

Components questions4. Student Choice: Topics/Options

Provided questions5. Response to Provided Information

questions

Scaffolded QuestionsScaffolded Questions

Scaffolded questions have multiple parts, Scaffolded questions have multiple parts, with each direction presented and labeled with each direction presented and labeled separately (e.g., A, B, C). separately (e.g., A, B, C).

The order is arranged so that successive The order is arranged so that successive questions depend upon the response to questions depend upon the response to the previous question. the previous question.

Often, each part becomes increasingly Often, each part becomes increasingly more difficult or complex.more difficult or complex.

Scaffolded question exampleScaffolded question example The framers of the U.S. Constitution wanted to prevent The framers of the U.S. Constitution wanted to prevent the new federal government from becoming a the new federal government from becoming a dictatorship. To keep the government from becoming too dictatorship. To keep the government from becoming too powerful, they divided its powers among three branches.powerful, they divided its powers among three branches.

a. For each of the three branches of government identify a. For each of the three branches of government identify one power given to it by the Constitution.one power given to it by the Constitution.b. Explain why each power you identified in part a is b. Explain why each power you identified in part a is important to our system of government. Support your important to our system of government. Support your answer with real-life examples.answer with real-life examples.

(Note: answering part b of this question requires that the (Note: answering part b of this question requires that the student be able to list branches of government in part a.)student be able to list branches of government in part a.)

Single Dimension/Component questions

Single Dimension/Component items ask a Single Dimension/Component items ask a straight-forward question which requires straight-forward question which requires explanation, examples, description, or explanation, examples, description, or evidence as support.evidence as support.

Single Dimension/Component exampleSingle Dimension/Component example

Rivers provide several advantages to Rivers provide several advantages to cities. Many Kentucky cities are located cities. Many Kentucky cities are located near large rivers. near large rivers.

Describe three important advantages that Describe three important advantages that the rivers provide these cities. Explain the rivers provide these cities. Explain why each advantage is important.why each advantage is important.

Two or More Relatively Two or More Relatively Independent ComponentsIndependent Components

Two or More Relatively Independent Two or More Relatively Independent Component items have at least two Component items have at least two directions (A, B, C, etc.) directions (A, B, C, etc.)

The directions may address the same The directions may address the same prompt but have little relation to each prompt but have little relation to each other. A correct response to one question other. A correct response to one question is not dependent upon the response to the is not dependent upon the response to the other questions.other questions.

Two or More Relatively Independent Two or More Relatively Independent Components exampleComponents example

Fossils provide important clues about things that Fossils provide important clues about things that have lived in the past.have lived in the past.

a. Describe a. Describe two two ways that fossils can form. ways that fossils can form. b. Explain b. Explain oneone way that fossils can help us way that fossils can help us understand how livings things have changed understand how livings things have changed over time.over time.

(Note: answering part b of this question does not (Note: answering part b of this question does not require the student be able to successfully require the student be able to successfully answer part a. The reverse is also true.)answer part a. The reverse is also true.)

Student Choice: Topics/Options Student Choice: Topics/Options ProvidedProvided

Student Choice items provide topics or Student Choice items provide topics or options that ask students to choose from options that ask students to choose from those options. those options.

They offer students more opportunities to They offer students more opportunities to demonstrate their individual learning, but demonstrate their individual learning, but may provide more scoring difficulty may provide more scoring difficulty because there are many more “correct” because there are many more “correct” answers.answers.

Student Choice exampleStudent Choice example

Some of Earth’s materials are listed below:Some of Earth’s materials are listed below:SoilSoil Water Water Gases of the atmosphereGases of the atmosphere RocksRocks

a. Choose TWO materials from the list. Explain a. Choose TWO materials from the list. Explain how a PLANT uses how a PLANT uses eacheach of these materials to of these materials to live.live.b. Choose TWO materials from the list. Explain b. Choose TWO materials from the list. Explain how an ANIMAL uses how an ANIMAL uses eacheach of these materials of these materials to live.to live.

Response to Provided InformationResponse to Provided Information

Students must be able to manipulate raw Students must be able to manipulate raw materials such as data, readings, or materials such as data, readings, or graphics in order to respond to specific graphics in order to respond to specific questions. questions. This question type is combined with This question type is combined with another type of question. another type of question. In the example on the next slide, the In the example on the next slide, the student is responding to a text passage, student is responding to a text passage, but the question is scaffolded as well. but the question is scaffolded as well.

Response to Provided Response to Provided Information exampleInformation example

Note: the student was required to read a text Note: the student was required to read a text passage before completing this questionpassage before completing this question

In the story “First Light,” Matthew woke up in In the story “First Light,” Matthew woke up in another time period, the 1850s.another time period, the 1850s.

a. Describe FOUR things Matthew discovered that a. Describe FOUR things Matthew discovered that were different from what he was used to in his were different from what he was used to in his present life.present life.b. Explain how EACH of those differences b. Explain how EACH of those differences affected him. Use information from the story to affected him. Use information from the story to support your answer.support your answer.

Steps in constructing Steps in constructing an open response questionan open response question

1. Decide1. Decide which Program of Studies, Core Content or other which Program of Studies, Core Content or other standard(s) you wish to assess.standard(s) you wish to assess.

2. Identify2. Identify the major concept(s) or learning target within the major concept(s) or learning target within these standards you want to assess. these standards you want to assess.

3. Decide3. Decide how deeply you wish to assess this concept-what how deeply you wish to assess this concept-what DOK level is acceptable for the question?DOK level is acceptable for the question?

4. Choose4. Choose the most appropriate question type from the five the most appropriate question type from the five possible choices.possible choices.

5. Establish5. Establish the situation by writing a question prompt that the situation by writing a question prompt that reflects the major concepts chosen.reflects the major concepts chosen.

6. Design6. Design directions that tell the student how to directions that tell the student how to

demonstrate knowledgedemonstrate knowledge 7. Create7. Create a question-specific scoring guide (rubric). a question-specific scoring guide (rubric).

Tips for Designing DirectionsTips for Designing Directions

Specify exactly what you want students to Specify exactly what you want students to answer. Specify numbers of responses if answer. Specify numbers of responses if appropriate. Specify three examples if you appropriate. Specify three examples if you require three, as opposed to ambiguous require three, as opposed to ambiguous terms like “some” or “several”.terms like “some” or “several”.Use simple and direct language. You are Use simple and direct language. You are evaluating what students know rather than evaluating what students know rather than attempting to determine if they can attempting to determine if they can decipher the question.decipher the question.


Use language that is both age and grade-Use language that is both age and grade-level appropriate. level appropriate.

Use simple, basic vocabulary when Use simple, basic vocabulary when appropriate, and technical vocabulary appropriate, and technical vocabulary when you are assessing the students’ when you are assessing the students’ knowledge of the knowledge of the meaning meaning of the technical of the technical word/phrase. word/phrase.

ExampleExample

When you plant a seed, the roots grow downward. This is called geotropism. Which factor is responsible for geotropism?

vs.When you plant a seed, the roots grow downward, and the stem grows upward. Which factor is responsible for the roots growing downward?


If the item asks students to read a If the item asks students to read a passage or examine a graphic and then passage or examine a graphic and then “give three ways” or “explain two reasons” “give three ways” or “explain two reasons” based on the material given, make sure based on the material given, make sure there are at least three ways or two there are at least three ways or two reasons found in the material provided.reasons found in the material provided.


If students are asked to generate a certain If students are asked to generate a certain number of ideas on their own without number of ideas on their own without having a passage or graphic to consult, having a passage or graphic to consult, then there should be at least then there should be at least twicetwice that that number of possible answers.number of possible answers.


If the students are required to respond to If the students are required to respond to multiple parts of a question, label each multiple parts of a question, label each part separately (A, B, C).part separately (A, B, C).

Developing a Scoring GuideDeveloping a Scoring Guide

Scoring guides are sets of criteria which Scoring guides are sets of criteria which describe the characteristics of responses describe the characteristics of responses at each identified level. at each identified level.

They provide the tool necessary to They provide the tool necessary to accurately evaluate student success for accurately evaluate student success for each individual question. each individual question.

Developing a Scoring GuideDeveloping a Scoring Guide

Scoring guides are developed following Scoring guides are developed following the actual writing of the question. the actual writing of the question.

This will allow the teacher to discover This will allow the teacher to discover potential problems with the question. potential problems with the question.

Constructing the scoring guide will also Constructing the scoring guide will also help ensure that the question is rich help ensure that the question is rich enough to support various levels of enough to support various levels of student responses. student responses.

A Scoring Guide Should…A Scoring Guide Should…

Include a clear explanation of what is Include a clear explanation of what is expected in a quality student response. expected in a quality student response. Define the various levels of possible Define the various levels of possible student responses and place a value on student responses and place a value on each level. each level. If there is more than one way a student If there is more than one way a student may achieve a given score level, the may achieve a given score level, the scoring guide should include those scoring guide should include those different possibilities.different possibilities.

A Scoring Guide Should…A Scoring Guide Should…

Enable scoring to be consistent, accurate Enable scoring to be consistent, accurate and as objective as possible. The scoring and as objective as possible. The scoring guide should provide a scorer with the guide should provide a scorer with the details necessary to score a response. details necessary to score a response. Use simple language and repeat Use simple language and repeat significant descriptive words used in the significant descriptive words used in the question. question. Ensure that what is required for a top-level Ensure that what is required for a top-level response is clearly indicated in the response is clearly indicated in the description. description.

The Kentucky General Scoring Guide The Kentucky General Scoring Guide

The Kentucky General Scoring Guide is a The Kentucky General Scoring Guide is a good example to follow when developing your good example to follow when developing your own scoring guides. own scoring guides. It utilizes four distinct performance levels and It utilizes four distinct performance levels and serves as a template for constructing an item-serves as a template for constructing an item-specific scoring guide. specific scoring guide. To download the general scoring guide: To download the general scoring guide: http://www.education.ky.gov/KDE/Administratihttp://www.education.ky.gov/KDE/Administrative+Resources/Testing+and+Reporting+/Distrive+Resources/Testing+and+Reporting+/District+Support/Tools+and+Resources/Test+Admict+Support/Tools+and+Resources/Test+Administration+Information.htmnistration+Information.htm

Establishing the parametersEstablishing the parameters

The first step in designing the scoring The first step in designing the scoring guide is to determine what a top-level guide is to determine what a top-level response should say. response should say.

In other words, what is the expectation for In other words, what is the expectation for a response that fully and completely a response that fully and completely answers the question? answers the question?


The next step is to write descriptions of The next step is to write descriptions of each of the other levels. each of the other levels. While there is no perfect formula for While there is no perfect formula for distinctions between levels, there should distinctions between levels, there should be appropriate and sequential differences be appropriate and sequential differences between levels. between levels. A good scoring guide helps make the A good scoring guide helps make the scorer’s task easier by clearly stating the scorer’s task easier by clearly stating the differences between levels in discernible differences between levels in discernible and important ways. and important ways.


Sometimes these distinctions will include Sometimes these distinctions will include quantity indicators. quantity indicators. For instance, if the question asks for three For instance, if the question asks for three examples, a response with two well-examples, a response with two well-defined examples might receive a score of defined examples might receive a score of three if you are using a four-level scoring three if you are using a four-level scoring guide. guide. Simple numerical indicators should not be Simple numerical indicators should not be the only difference between levels; quality the only difference between levels; quality of work must also be considered. of work must also be considered.


KCCT scoring guides are written with four KCCT scoring guides are written with four levels of performance, as well as zero or levels of performance, as well as zero or blank non-performance levels. blank non-performance levels.

Common descriptors used to distinguish Common descriptors used to distinguish the four performance levels are listed on the four performance levels are listed on the slides that follow:the slides that follow:

4 Response 4 Response

The scoring guide for a four response The scoring guide for a four response typically characterizes responses as typically characterizes responses as effective, thorough, complete, successful, effective, thorough, complete, successful, insightful, in-depth, efficient, and/or insightful, in-depth, efficient, and/or sophisticated. sophisticated.

3 Response3 Response

A three response will usually use terms A three response will usually use terms such as adequate, satisfactory, such as adequate, satisfactory, understanding of major concepts, understanding of major concepts, complete most, and/or clear. complete most, and/or clear.


A response of two will often be described A response of two will often be described as having gaps or leaps, incomplete, some as having gaps or leaps, incomplete, some important points, demonstrate basic important points, demonstrate basic understanding, and/or some errors. understanding, and/or some errors.


Responses receiving a one are typically Responses receiving a one are typically labeled as minimal, completes only small labeled as minimal, completes only small part, little understanding, not logical, part, little understanding, not logical, unclear, and/or major errors. unclear, and/or major errors.


Finally, review your scoring guide to Finally, review your scoring guide to confirm that it is consistent with both the confirm that it is consistent with both the question and the standard(s) it is intended question and the standard(s) it is intended to assess. to assess.

Reminders:Reminders:As you design your scoring guide, As you design your scoring guide, remember to ask yourself: remember to ask yourself:

What does a 4 response contain? What does a 4 response contain? Can I write a 4 response? Can I write a 4 response? Is my guide consistent with the question, Is my guide consistent with the question, Academic Expectations, Program of Academic Expectations, Program of Studies, Core Content and unit Studies, Core Content and unit instruction? instruction? Is each level clearly different from other Is each level clearly different from other levels? levels?

Common misconceptions about scoring Common misconceptions about scoring of open response items on the KCCTof open response items on the KCCT

Misconception:Misconception:

Restating the question is mandatory.Restating the question is mandatory.

Fact:Fact:

No, it is not required and doing so will not No, it is not required and doing so will not gain additional points. gain additional points.



Responses restating the question without Responses restating the question without further information will be given at least further information will be given at least one point. one point.

Fact:Fact:

No, additional information must be No, additional information must be included in order to receive any credit. included in order to receive any credit.


Misconception:Misconception:A graphic organizer should be done on the A graphic organizer should be done on the response page. response page.

Fact:Fact:Depending on the type of question being Depending on the type of question being asked a graphic organizer may not be the asked a graphic organizer may not be the best way to record the answer. Best best way to record the answer. Best practice would be to create the organizer practice would be to create the organizer on scrap paper and then record. on scrap paper and then record.


Misconception:Misconception:Answers must be in paragraph form. Answers must be in paragraph form.

Fact:Fact:Scorers are trained to focus on content Scorers are trained to focus on content and not address the format of the and not address the format of the response. A response in any format: response. A response in any format: bulleted, labeled diagram, or graphic bulleted, labeled diagram, or graphic organizer, will be scored. organizer, will be scored.


Misconception:Misconception:Doing more than required by the prompt Doing more than required by the prompt will score a 4. will score a 4.

Fact:Fact:A 4 will be assigned to a response that A 4 will be assigned to a response that completely and accurately reflects the completely and accurately reflects the correct answer according to the rubric. No correct answer according to the rubric. No additional information is required to score additional information is required to score a 4. a 4.


Misconception:Misconception:Must use content specific vocabulary in Must use content specific vocabulary in order to score a 4. order to score a 4.

Fact:Fact:Not necessarily, if the content can be Not necessarily, if the content can be adequately expressed without the use of adequately expressed without the use of specific vocabulary appropriate credit will specific vocabulary appropriate credit will be given to the response. be given to the response.


Misconception:Misconception:Three or more examples must always be Three or more examples must always be given. given.

Fact:Fact:No, the question will specify the number of No, the question will specify the number of examples required. Giving more will not examples required. Giving more will not increase the odds of receiving a higher increase the odds of receiving a higher score. score.



Scorers only have 30 seconds to score Scorers only have 30 seconds to score each piece. each piece.

Fact:Fact:

No, scorers can take as much time as No, scorers can take as much time as needed on each piece. needed on each piece.

Writing Multiple Choice QuestionsWriting Multiple Choice Questions

Characteristics of a KCCT-like Characteristics of a KCCT-like Multiple Choice Item:Multiple Choice Item:

Directly tied to one or more content Directly tied to one or more content standardsstandards

Consists of a stem (statement or question) Consists of a stem (statement or question) and response selections (correct response and response selections (correct response and distractors)and distractors)

Characteristics of a KCCT-like Characteristics of a KCCT-like Multiple Choice Item:Multiple Choice Item:

MOST of Earth is covered by

land. water. trees. desert.

Stem

Key/Correct Answer Distractors

Steps in Constructing a Steps in Constructing a Multiple Choice QuestionMultiple Choice Question

1. Decide1. Decide which Program of Studies, Core Content which Program of Studies, Core Content or other standard(s) you wish to assess.or other standard(s) you wish to assess.

2. Identify2. Identify the major concept(s) or learning target the major concept(s) or learning target within these standards you want to assess. within these standards you want to assess.

3. Decide3. Decide how deeply you wish to assess this how deeply you wish to assess this concept-what DOK level is acceptable for the concept-what DOK level is acceptable for the question? question?

4. 4. Design Design the stem or question. the stem or question.

5. Design 5. Design the key and distractors.the key and distractors.

Additional Design Additional Design ConsiderationsConsiderations

The item stems should be stated in positive The item stems should be stated in positive terms as much as possible. terms as much as possible. The use of negatives (e.g., “not”) in The use of negatives (e.g., “not”) in bothboth the the item stem and the answer choices is very item stem and the answer choices is very confusing.confusing.Avoid the use of absolute terms (e.g., always, Avoid the use of absolute terms (e.g., always, never, all, none, only) in the distractors as much never, all, none, only) in the distractors as much as possible. as possible. Whenever possible, avoid answer choices that Whenever possible, avoid answer choices that are mutually exclusive opposites (e.g., are mutually exclusive opposites (e.g., living/non-living, fiction/nonfiction). living/non-living, fiction/nonfiction).


Avoid "Avoid "what do you thinkwhat do you think . . ." because any . . ." because any answer will have to be considered correct.answer will have to be considered correct.

Try to keep the stem shorter than 4 sentences. Try to keep the stem shorter than 4 sentences.

List alternatives vertically beneath the stem, List alternatives vertically beneath the stem, beginning a new line with each response.beginning a new line with each response.

Have students circle the letter beside their Have students circle the letter beside their response to avoid scoring problems reading response to avoid scoring problems reading student handwriting.student handwriting.


Number the questions (stems), and use capital Number the questions (stems), and use capital or lowercase letters for the responses. or lowercase letters for the responses. The position (A, B, C, etc.) of the correct answer The position (A, B, C, etc.) of the correct answer should vary from question to question so there is should vary from question to question so there is no pattern that could lead to predicting the no pattern that could lead to predicting the correct answer.correct answer.The stem and answer choices should appear on The stem and answer choices should appear on the same page.the same page.Avoid using language in the question that might Avoid using language in the question that might accidentally lead students to favor an answer accidentally lead students to favor an answer based on language alone.based on language alone.


When referring to a map, table, or figure, label it When referring to a map, table, or figure, label it for easy reference and develop an introductory for easy reference and develop an introductory sentence about the graphic that precedes it on sentence about the graphic that precedes it on the page. the page. One question should not give a clue to the One question should not give a clue to the answer of another question.answer of another question.Avoid composing alternatives in which there are Avoid composing alternatives in which there are only microscopically fine distinctions between only microscopically fine distinctions between the answers, unless the ability to make these the answers, unless the ability to make these distinctions is the primary target or standard distinctions is the primary target or standard being assessed.being assessed.

Find the flaw:Find the flaw:

A. in the kitchen

B. on the cutting boards

C. on the stove-top burners*

D. in hot ovens

According to the passage, where do most home accidents occur?

A. Gothic.

B. modern.

C. Romanesque.

D. Byzantine.*

A style of architecture that uses the Roman arch, thick walls, and dimly lit interiors is called

A. for reconciliation of faith and reason.

B. that the ends justify the means.*

C. for the rise of the common man.

D. for allowing freedom of thought.

In The Prince, Renaissance author, Machiavelli, instructs the ruling monarchy in methods of statesmanship and argues

A. increase the frog population and be harmful to the grass in the area.*

B. be harmful to both frogs and grass. C. have little effect on the plants in the area.

D. cause most animals to leave the area.

Some students want to build a pond near their schoolyard.

This change will MOST LIKELY

Which is a reason Mike went to the park?Which is a reason Mike went to the park?

A.A. He wanted to play with his friends.He wanted to play with his friends.

B.B. He did not want to go to school.He did not want to go to school.

C.C. He liked playing on the swings.He liked playing on the swings.

D.D. He hoped to find his notebook.He hoped to find his notebook.

Janice can Janice can bestbest be described as be described as

A.A. angry.angry.

B.B. mean.mean.

C.C. rude.rude.

D.D. caring.caring.

Color is to art as _______ is to music.

A. melody

B. rhythm

C. dynamics

D. timbre*

To conduct an experiment, Mike pulls marbles from a bag that contains 100 marbles. Each time he pulls a marble he records the color and then returns it to the bag. He does this 10 times. His results are shown below. Theoretically, how many red marbles and blue marbles are in the bag? Red / Blue

A. 60 red, 40 blue

B. 50 red, 50 blue

C. 70 red, 30 blue*

D. 30 red, 70 blue