Activities: Writing Questions

Clicker WorkshopsUniversity of Colorado Science Education Initiativehttp://colorado.edu/sei

ddddddddd

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

OverviewThese activities give participants a chance to try writing and revising clicker questions, with feedback from peers or group leader.

ObjectivesThe goal is for participants to learn the features of good clicker questions, and get a chance to practice these skills where they can get formative feedback. Participants should be able to identify the cognitive level, or “depth” of a question, as well as its’ pedagogical goal.

Activities1. Write a question based on a pedagogical goal

Participants choose a pedagogical goal from the Question Cycle handout (see below); e.g., “predict outcome.” They draft a clicker question related to their content area, including some plausible distractors, to fulfill this pedagogical goal. This activity occurs before discussing features of effective questions; this draft question is intended to serve as a “touchstone” during the rest of the workshop, as participants shop for ideas to improve it and later revise the question based on what they have learned. Then, after discussing features of good clicker questions, participants work on revising their question and share with a neighbor.

Note that this only works well in workshops focused on question-writing; otherwise instructors don’t want to take the time for this activity.

Variation #1: Instead, do this activity after a discussion of features of effective questions, and have participants then swap with a neighbor.

Variation #2: Have participants brainstorm a question in groups of 2-3, and then swap with another group to revise each others’ questions.

Variation #3: Draft a question on a topic of their choice, and then determine the pedagogical goal of that question after the fact.

Materials1. Write question based on

pedagogical goal: Handout, below, including Question Cycle, and Question Writing Tips. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.

2. Write question based on learning goal: Handout, below, including Question Writing Tips. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.

3. Revise and improve existing question: Handout, below.

4. Rate and Swap: 5-10 mins. Handout, below, including one of Bloom’s Taxonomy handouts. Useful to have small whiteboards or butcher paper and markers so that questions can be written large enough to share.

Time1. Write question based on

pedagogical goal: 5-10 minutes to write, 10 mins to revise.

2. Write question based on learning goal: 5-10 minutes to write, 10 mins to revise

3. Revise and improve existing question: 5-10 mins

4. Rate and Swap: 5-10 mins.

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Activities: Writing Questionsddddddddd

2. Write a question based on a content learning goal

Same as above, but participants choose a learning goal related to their content area. Here are some example learning goals in the sciences:

Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell

Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).

Chemistry: Explain trends in boiling points in terms of intermolecular interactions

Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.

Math: Solve a system of linear equations in two variables using algebra or graphing.

3. Try to revise and improve an existing question

It can be difficult in a short workshop for participants to really engage with the process of writing and revising their own question; examining an existing question can be an easier strategy. This is challenging with a multidisciplinary audience, however. You might give each disciplinary group their own question (from the Gallery Walk, for example).

4. Rate and swap

Participants are given a handout of Bloom’s Taxonomy levels and verbs. Using the question that they wrote, they rate it on Bloom’s Taxonomy, swap with a neighbor to come to consensus on its’ Bloom’s level, and then try to write a question at a higher level of Bloom’s using the verbs on the Bloom’s Taxonomy sheet. Note: It is often difficult to Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

identify a single Bloom’s level for a question, typically it will span two categories.

Variation: Have participants rate the Bloom’s level of a pre-written question, and attempt to “Bloomify it up.”

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Write a Draft Question: Pedagogical Goal

To DoThink about a topic you will be teaching next week (or that you recently taught). On your own, write a draft question that addresses one of the pedagogical goals from the Question Cycle. Include 3-4 plausible distractors.This doesn’t need to be perfect – we will work on revising it later in the workshop.If you finish, swap with your neighbor and discuss.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Write a Draft Question: Content GoalTo DoLook at the learning goals below. On your own, write a draft question that addresses this learning goal. Include 3-4 plausible distractors.

Biology: Recognize the components of a cell and describe why each is necessary for the function of a cell

Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).

Chemistry: Explain trends in boiling points in terms of intermolecular interactions

Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.

Math: Solve a system of linear equations in two variables using algebra or graphing.

Then, consider the pedagogical goals from the Question Cycle: What is the pedagogical goal of this question?This doesn’t need to be perfect – we will work on revising it later in the workshop.If you finish, swap with your neighbor and discuss.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Rate & SwapTo DoUse the Bloom’s Taxonomy handout (below) to rate the Bloom’s level of the question that you wrote. (You can indicate that it belongs to more than one level).Swap your question with a neighbor. Do you agree on the Bloom’s level of your question?Try to “Bloomify up” the level of your question – rewrite it at a higher level of Bloom’s, making use of the verbs associated with each level on the Bloom’s handout.

Notes

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Writing Questions Courtesy Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Writing Questions Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Tips for Writing Clicker Questions

See also the Instructor’s Guide to the Effective Use of Clickers, at http://STEMclickers.colorado.edu

Start with existing questions where possible: See lists of question banks at http://STEMclickers.colorado.edu.

Don’t agonize too much; after all, the perfect question doesn’t solve all problems: It’s hard to tell in advance which questions will be great, and a great question poorly facilitated can fall flat.

Don’t make them too easy. This is a common mistake, and misleads students as to your expectations. Challenge student thinking (students prefer this!) rather than testing memorized facts. You can write multiple choice questions that test higher levels of thinking (e.g, Analyzing, Evaluating; see Bloom’s Taxonomy)!

Use questions that emphasize reasoning or process rather than the correct answer; this is knowledge that is more generalizeable.

Use questions that will prompt discussion. Interesting questions that students can’t answer on their own are more likely to spur productive discussion.

Use clear wording so that students understand what they are being asked. Keep revising over time.

Write tempting distractors using your knowledge of student difficulties. For example, look at student answers on exams or quizzes, or first give the question as an open-ended question to generate common wrong answers. Talk to other instructors who have taught the course in the past, and talk to students one-on-one in office hours.

Use a wide variety of creative question types. You can survey your students on their experiences or beliefs, assess what students already know about a topic, ask them to reflect on their own understanding, ask students to predict an outcome, use a series of questions to break problems into parts, stimulate discussion, or use pictures or graphs in the answer choices.

Writing Questions Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Good sources of questions: Questions your students ask you or that you overhear Common analogies you use as a teacher A series of connected questions to lead students through

reasoning Interpret graphs, data, pictures, etc. Discussion questions where there is no one right answer

Bloom’s Taxonomy: Cognitive Levels and Verbs

(short handout #1)

Knowledge Comprehension Application Analysis Evaluation Synthesis

knowdefine

memorizelist

recallnamerelate

restatediscussdescriberecognize

explainidentifylocate

translateinterpret

applyemploy

demonstratedramatizepracticeillustrateoperate

distinguishanalyze

differentiatecalculate

experimentcomparecontrastcriticizesolve

examine

composeplan

proposedesign

assembleconstruct

createdesign

organizemanage

judgeappraiseevaluatecompare

valueselect

chooseassess

estimatemeasure

Writing Questions Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Bloom’s Taxonomy: Cognitive Levels and Verbs

(short handout #2)

Writing Questions Science Education Initiative at the University of Colorado; http://colorado.edu/sei

Bloom's Taxonomy “Revised”Key Words, Model Questions, & Instructional Strategies

Bloom’s Taxonomy (1956) has stood the test of time. Recently Anderson & Krathwohl (2001) have proposed some minor changes to include the renaming and reordering of the taxonomy. This reference reflects those recommended changes.

I. REMEMBER (KNOWLEDGE) (shallow processing: drawing out factual answers, testing recall and recognition)

Verbs for Objectives Model Questions Instructional Strategieschoosedescribedefineidentifylabellistlocatematchmemorizenameomitreciterecognizeselectstate

Who?Where?Which One?What?How?What is the best one?Why?How much?When?What does It mean?

HighlightingRehearsalMemorizingMnemonics

II. UNDERSTAND (COMPREHENSION) (translating, interpreting and extrapolating)

Verbs for Objectives Model Questions Instructional Strategiesclassifydefenddemonstratedistinguishexplainexpressextendgive exampleillustrateindicateinterrelateinterpretinferjudgematchparaphraserepresentrestaterewriteselectshowsummarizetell

State in your own words.Which are facts?What does this mean?Is this the same as. . .?Give an example.Select the best definition.Condense this paragraph.What would happen if . . .?State in one word . . .Explain what is happening.What part doesn't fit?Explain what is meant.What expectations are there?Read the graph (table).What are they saying?This represents. . .What seems to be . . .?Is it valid that . . .?What seems likely?Show in a graph, table.Which statements support . . ?What restrictions would you add?

Key examplesEmphasize connectionsElaborate conceptsSummarizeParaphraseSTUDENTS explainSTUDENTS state the rule“Why does this example. . .?”create visual representations (concept maps, outlines, flow charts organizers, analogies, pro/con grids) PRO| CONNOTE: The faculty member can show them, but they have to do it.Metaphors, rubrics, heuristics

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006

translate

III. APPLY

(Knowing when to apply; why to apply; and recognizing patterns of transfer to situations that are new, unfamiliar or have a new slant for students)

Verbs for Objectives Model Questions Instructional Strategiesapplychoosedramatizeexplaingeneralizejudgeorganizepaintprepareproduceselectshowsketchsolveuse

Predict what would happen ifChoose the best statements that applyJudge the effectsWhat would resultTell what would happenTell how, when, where, whyTell how much change there would beIdentify the results of

ModelingCognitive apprenticeships“Mindful” practice – NOT just a “routine” practicePart and whole sequencingAuthentic situations“Coached” practiceCase studiesSimulationsAlgorithms

IV. ANALYZE (breaking down into parts, forms)

Verbs for Objectives Model Questions Instructional Strategiesanalyzecategorizeclassifycomparedifferentiatedistinguishidentifyinferpoint outselectsubdividesurvey

What is the function of . . .?What's fact? Opinion?What assumptions. . .?What statement is relevant?What motive is there?Related to, extraneous to, not applicable.What conclusions?What does the author believe? What does the author assume?Make a distinction.State the point of view of . . .What is the premise?State the point of view of . . .What ideas apply?What ideas justify the conclusion?What's the relationship between?The least essential statements areWhat's the main idea? Theme?What inconsistencies, fallacies?What literary form is used?What persuasive technique?Implicit in the statement is . . .

Models of thinkingChallenging assumptionsRetrospective analysisReflection through journalingDebatesDiscussions and other collaborating learning activitiesDecision-making situations

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006

V. EVALUATE (according to some set of criteria, and state why)

Verbs for Objectives Model Questions Instructional Strategiesappraisejudgecriticizedefendcompare

What fallacies, consistencies, inconsistencies appear?Which is more important, moral, better, logical, valid, appropriate?Find the errors.

Challenging assumptionsJournalingDebatesDiscussions and other collaborating learning activitiesDecision-making situations

VI. CREATE (SYNTHESIS) (combining elements into a pattern not clearly there before)

Verbs for Objectives Model Questions Instructional Strategieschoosecombinecomposeconstructcreatedesigndevelopdoformulatehypothesizeinventmakemake uporiginateorganizeplanproducerole playtell

How would you test. . .?Propose an alternative.Solve the following.How else would you . . .?State a rule.

ModelingChallenging assumptionsReflection through journalingDebatesDiscussions and other collaborating learning activitiesDesignDecision-making situations

Web References: http://www.coun.uvic.ca/learn/program/hndouts/bloom.html http://www.fwl.org/edtech/blooms.html http://apu.edu/~bmccarty/curricula/mse592/intro/tsld006.htm http://152.30.11.86/deer/Houghton/learner/think/bloomsTaxonomy.html http://amath.colorado.edu/appm/courses/7400/1996Spr/bloom.html http://www.stedwards.edu/cte/bloomtax.htm http://quarles.unbc.edu/lsc/bloom.html http://www.wested.org/tie/dlrn/blooms.html http://www.bena.com/ewinters/bloom.html http://weber.u.washington.edu/~krumme/guides/bloom.html

References: Anderson, L. W. & Krathwohl, D. R. (2001). A Taxonomy for learning, teaching, and assessing. Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals, by a committee of college and university examiners. New York: Longmans.John Maynard, University of Texas, AustinMarilla Svinicki, University of Texas, Austin

Compiled by the IUPUI Center for Teaching and Learning, Revised December 2002

www.center.iupui.edu/ctl/idd/docs/Bloom_revised021.doc, February 8, 2006