Workshop on Peer Instruction: Making Good Clicker Questions Beth Simon Computer Science and Engineering University of California, San Diego Credits: Sara

Workshop on Peer Instruction: Making Good Clicker Questions

Beth SimonComputer Science and EngineeringUniversity of California, San Diego

Credits: Sara Harris, Earth and Ocean Sciences, UBC Clicker Resource Guide from CU-SEI and CWSEI Rosie Redfield (for clicker question contribution)

Roland Stull (for clicker question contribution)

By the end of today’s talk you should be able to…

• LG1: Describe your reasons for wanting to use PI and how that informs what you should plan to do

• LG2: Identify your concerns about using PI and present pros and cons associated with it

• LG3: Refer back to the Clicker Resource Guide when creating clicker questions

• LG4: Analyze a clicker question for it’s Bloom’s taxonomy level

What are your primary concerns in your current course?

1.

2.

3.

4.

5.

Which of those do you think Peer Instruction will help?

• Discuss

What concerns do you have about using Peer Instruction?

1.

2.

3.

4.

5.

End-of-Term survey, upper-level science course (200 students)University of British Columbia

What do students think?

What do you think worked well in this course, and why?

# Responses

“clicker questions helped me understand major concepts and how to apply the material”

“it gave me a sense of what I understood and what I didn't.”

“clicker questions were really helpful because they made me to think about the material during class.”

“I liked the clicker questions because they made me think deeper into particular concepts, but I did find them very challenging.”

Stages of clicker use

1. Stage 1 – asking simple, primarily factual questions.

2. Stage 2 – asking more challenging conceptual questions, or questions where the answer is not obvious and critical points could be argued.

--Perhaps after short mini-lectures (10 min)

3. Stage 3 – Lecture is structured around a set of challenging clicker questions that largely embody the material students are to learn.

More Detail:Clicker Resource Guide

• Executive Summary

• Overview the rest– So you know what you can return to

Types of Clicker Questions

• What are you going to ask?

• What do you want them to know?

Bloom’s taxonomy VERBS for Stage 1(knowledge)

KnowDefineMemorizeRepeatRecordListRecallNameRelate

Level 1 example

Who is/was the 12th president of UBC?

A. Martha PiperB. David StrangwayC. Leonard S. KlinckD. Frank F. WesbrookE. Stephen Toope

Bloom’s taxonomy VERBS for Stage 2(Comprehension, Application)

TranslateInterpretApplyEmployUseDemonstrateDramatizePracticeIllustrateOperateSketch

DiscussDescribeRecognizeExplainExpressIdentifyLocateReportReviewTellRestate

B > b > bl

B– black

bb or bb' brown

b'b' cinnamon

Two black cats are crossed.

Which of the following outcomes is not possible?

A. Only black kittens.

B. Black and brown kittens.

C. Black and cinnamon kittens.

D. Black, brown and cinnamon kittens.

E. All are possible.

Level 2 example

Bloom’s taxonomy VERBS for Stage 3(Analysis, Synthesis, Evaluation)

ComposePlanProposeDesignFormulateAssembleConstructCollectCreateDesignOrganize

DistinguishAnalyzeDifferentiateAppraiseCalculateCompareContrastDiagramDebateRelateExamine

JudgeAppraiseEvaluateRateValueReviseScoreSelectChooseEstimateMeasure

Level 3 example

How to write a clicker question:

1. Pick a concept or skill you think is important and/or difficult for students.

2. If you know them, identify any common misconceptions to use as distractors.

3. Write the question as simply and clearly as possible.

There’s lots of literature about writing effective MC questions. It’s not easy. Here’s a link to get started with the issues around MCQs: http://teambasedlearning.apsc.ubc.ca/?page_id=163

http://teambasedlearning.apsc.ubc.ca/?page_id=163

How to respond to histograms:

What would you do if you got this histogram?

A. B. C. D. E.

B C D EA

How to respond to histograms:

Example: Why do we have seasons on Earth?

A. Because the Earth is spinningB. Because the Earth is tiltedC. Because the Earth is sphericalD. Because the Earth is sometimes closer, sometimes farther from the SunE. Because the Earth’s equator is closer to the Sun than the poles

Chemistry: Equilibrium

64. (Equilibrium, Le Châtelier's principle; Ch. 5 & 9 "Companion") The body-centered cubic (bcc) phase of a metal has a 68% packing efficiency, while the face-centered cubic (fcc) phase of the same metal has a 74%packing efficiency. For the equilibriumbcc metal = fcc metal

as pressure increases, the equilibrium

A)shifts to the leftB)shifts to the rightC)is unaffected

Chemistry: Kinetics and Mechanism

129. (Catalysts) Which curve illustrates the effect of a catalyst on the reaction diagram, given that it speeds up the rate of a reaction?

ABCD

How many times is each set of code executed?

Pixel[] pixelArray = this.getPixels();int value = 0;Pixel p = null;for(int index = 0; index < pixelArray.length; index++);{ p = pixelArray[index]; value = p.getRed(); value = (int) (value * 0.5); p.setRed(value);}

1) Solo: (30 sec)2) Discuss: (1min)3) Group: (30 sec)

How many times is each set of code executed?

Pixel[] pixelArray = this.getPixels();int value = 0;Pixel p = null;for(int index = 0; index < pixelArray.length; index++);{ p = pixelArray[index]; value = p.getRed(); value = (int) (value * 0.5); p.setRed(value);}


What picture most accuratelydescribes what this code does ?

Pixel[] pixelArray = this.getPixels();int value = 0;Pixel p = null;for(int index = 0; index < pixelArray.length; index++){ p = pixelArray[index]; q = pixelArray[index+1]; p.setRed(q.getRed()); p.setBlue(q.getRed()); p.setGreen(q.getGreen());}


What picture most accuratelydescribes what this code does ?



• A method in Picture.java… Pixel p;for (int foo = 0; foo < getWidth(); foo++){ for (int bar 0; bar < getHeight(); bar++) { p = getPixel(foo, bar); p.setColor(Color.BLACK); }}


Nested Loops: How do they work?What order are pixels changed?

• A method in Picture.java… Pixel p;for (int foo = 0; foo < getWidth(); foo++){ for (int bar 0; bar < getHeight(); bar++) { p = getPixel(foo, bar); p.setColor(Color.BLACK); }}


Nested Loops: How do they work?What order are pixels changed?

Why does this codehave an error?

A. It tries to access pixelArray[-1]B. It tries to access pixelArray[0]C. It tries to access pixelArray[pixelArray.length]D. It tries to access pixelArray[pixelArray.length+1]E. None of the above



Why does this codehave an error?

A. It tries to access pixelArray[-1]B. It tries to access pixelArray[0]C. It tries to access pixelArray[pixelArray.length]D. It tries to access pixelArray[pixelArray.length+1]E. None of the above



DEBUGGING: Where is the best place to put a print statement?

Pixel[] pixelArray = this.getPixels();int value = 0;int index = 0;while (index < pixelArray.length){ value = pixelArray[index].getRed(); value = (int) (value * 0.5); pixelArray[index].setRed(value); index = index + 1;}


DEBUGGING: Where is the best place to put a print statement?

Pixel[] pixelArray = this.getPixels();int value = 0;int index = 0;while (index < pixelArray.length){ value = pixelArray[index].getRed(); value = (int) (value * 0.5); pixelArray[index].setRed(value); index = index + 1;}


SPEEDY:What does that code do?

A. Makes it higher pitchedB. Makes it louderC. Makes if lower pitchedD. Makes it quieterE. Makes a silent sound

public void funky(){ SoundSample[] noiseArray = this.getSamples(); int zzz = 0; for (int i=0;i<noiseArray.length) { SoundSample sample = noiseArray[i]; int foo = sample.getValue(); zzz += foo; } int yyy = zzz / noiseArray.length; for (int i = 0; i < noiseArray.length; i++) { SoundSample sample = noiseArray[i]; sample.setValue(yyy); }}

1) Solo: (20 sec)2) Discuss: (1 min)3) Group: (20 sec)

SPEEDY:What does that code do?

A. Makes it higher pitchedB. Makes it louderC. Makes if lower pitchedD. Makes it quieterE. Makes a silent sound

public void funky(){ SoundSample[] noiseArray = this.getSamples(); int zzz = 0; for (int i=0;i<noiseArray.length) { SoundSample sample = noiseArray[i]; int foo = sample.getValue(); zzz += foo; } int yyy = zzz / noiseArray.length; for (int i = 0; i < noiseArray.length; i++) { SoundSample sample = noiseArray[i]; sample.setValue(yyy); }}

1) Solo: (20 sec)2) Discuss: (1 min)3) Group: (20 sec)

Best Practice Quick Sheet• Give reading quizzes before each lecture

• Foster group discussion

– Assign groups (and keep the same ones all term)

– Require them to reach consensus and vote “the same”

– (Rotate ownership of discussion within groups)

• Ask challenging questions

– That make for meaningful discussion

– 35-75% solo -> >75% group

• Respond to student experience

– Ask them to provide explanations first (then you can re-word)

– If the group vote correctness is low• Take time to review, explain, clarify

• Frequently (once a week, then often) remind students of the value of Peer Instruction

Documents

Workshop on Peer Instruction: Making Good Clicker Questions Beth Simon Computer Science and Engineering University of California, San Diego Credits: Sara