Interactive simulations for teaching Interactive simulations for teaching physics; a powerful (and dangerous) physics; a powerful (and dangerous) educational tooleducational tool

Work supported by: NSF, Hewlett Foundation, Univ. of Colorado, me and Sarah

Carl WiemanUBCCU

Physics Education Technology Project (PhET)Develop interactive simulationsResearch on simulation design and effectiveness

Goals for talk Examples of good simulationsResearch results on sim design and effectiveness andhow matches other research on learning.

When simulations carefully tested and refined :•Highly engaging•Very effective for learning•Work with very wide range of students (“grade school to grad school”)

PhET (phet.colorado.edu)•~ 60 interactive simulations •Intro physics, modern physics, some chemistry, bit of math,starting to expand into geo and bio, …•Phet-based activities database on website

show website, sim list, balloons and sweater, moving man, gas

run phet sims (all free!): • directly from web (regular browser, platform independent)• download whole website to local computer for offline use

2006-- 1 Million sims launched off website; 50,000 full site downloads

Extensive development and testing process--teams (faculty, software engineers, sci. ed. specialists)

Physics faculty:Physics faculty: Michael DubsonMichael DubsonNoah FinkelsteinNoah FinkelsteinKathy Perkins (manager)Kathy Perkins (manager)Carl Wieman Carl Wieman

Postdocs: Postdocs: Sam McKaganSam McKaganLinda Koch (Chem)Linda Koch (Chem)

Software Engineers: Software Engineers: Ron LeMasterRon LeMasterSam ReidSam ReidChris MalleyChris MalleyMichael Dubson Michael Dubson

Grad students:Grad students:Wendy AdamsWendy AdamsDanielle HarlowDanielle HarlowChris KellerChris KellerNoah PodolefskyNoah Podolefsky

HS Teacher:HS Teacher: Trish LoebleinTrish Loeblein

~6 full time equivalentsStaff: Staff:

Mindy GratneMindy Gratney,y, Linda Wellmann Linda Wellmann

Phet Staff

• Engaging and productively fun

(interface design, appearance, …) • Connection to real world • Highly interactive- stuff happening, user controls• Explicit visual & conceptual models (experts’)• Explore and discover, with productive constraints

Achieve deep understanding

K.K. Perkins, et al, “PhET: Interactive Simulations for Teaching and Learning Physics”, Physics Teacher (Jan 2006)

Design Features and Criteria

Most important element--testing with students

1. Think aloud interviews (~250 hours) Explore with guiding question

2. Study effectiveness in• Lecture- demo replacement. (x 3 improvement!)• Lab (replacement or supplement) (better than real)• Homework

many references on research available on phet website

Experts- - really like.

Students--Watch without interacting. Don’t like. Misinterpret.

Example- of what revealed by interviewstudies.

Radio waves.Initial startup.

Matches research on learning.•Cognitive demand. Novices don’t know what to focus on.treat everything equally important, overwhelming

• Construction of understanding.(animation gets attention, but achieves little learning)•Developing expert organization/categorization.

Why starting this way worksso much better?

Example illustrates important principle:students think and perceive differently from experts

Good teaching is presenting material so novice students learn from it, not so looks good to experts!Violated by most simulations

(and many lecture demonstrations, figures in texts,…)

Interesting results from interview studies (cont.)

the good, the bad, and the evil simGood sim is extremely effective for wide range of students: understand difficult concepts, can explain & apply to real world situations.

Bad sim- very little learned.Awkward distracting interface, boring, confusing, ornoninteractive (animation gets attention, not learning)

Evil sim--effective at teaching wrong things!

Only safe to use sims that have been tested with real students!

Pre-class or pre-lab Activity

Lecture/classroom Visual Aids, Interactive Lecture

Demos, & Concept tests

Labs/Recitations Group activities

Homework bits of examples of effectiveness in different settings

Sims useful in variety of settings

1. When the string is in position B, instantaneously flat, the velocity of points of the string is...A: zero everywhere. B: positive everywhere.C: negative everywhere. D: depends on the position.

AB

C

snapshots at different times.

Wave-on-string sim vs Tygon tube demo

2. At position C, the velocity of points of the string is...A: zero everywhere. B: positive everywhere.C: negative everywhere. D: depends on the position.

Correct :2002 demo: 27%2003 sim: 71%

Correct :2002 demo: 23 %2003 sim: 84%

Follow-up Concept Tests:

Lecture (Non-science Majors Course)

Standing waves-- Sim vs. Demonstration

show wave on string

Features that make a difference- experts hardly notice, BIG difference for novices

1. Green beads on string that show moves up and down, not sideways.

2. Speed set so novice brain can absorb and makesense of it.

3. Can do controlled changes, most in response to studentrequests. Sort out what makes a difference and why.

Integrating a sim on a topic (Lect. & HW)(Photoelectric Effect in Modern Physics) (S. McKagan, to be pub.)

Course Q1 Q2 Q3 N

UW w/o PT 65 40 20 26

UW w/ PT 75 85 40 36

% Correct

Univ. of Wash.: • Student learning of photoelectric effect deficient• Developed & used Photoelectric Tutor (PT)

Exam Q: What would happen to current reading if you:Q1: Changed metal. Why?Q2: Double intensity of light. Why?Q3: Increased V across electrodes. Why?

CU: Incorporated sim

CU Fa05 91 87 85 189

CU Sp06 86 88 84 182

show photoelectric effect

What does simulation provide over usual figure and explanation in textbook?(to expert very little, to student, a lot!)

1. simplified circuitry, connect to familiar (light, battery), specific items research showed as problems

2. see direct cause and effect relations

3. explicit visual model

explicit visual model-- example 2. cck sim

N D. Finkelstein, et al, “When learning about the real world is better done virtually: a study of substituting computer simulations for laboratory equipment,” PhysRev: ST PER 010103 (Sept 2005)

DC Circtuis Exam Questions

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

q1 q2 q3 cntlQuestion

Fra

cti

on

Co

rrect

CCK (N =99)

TRAD (N=132)

p < 0.001

DC Circuit Final Exam Questions

Standard Laboratory (Alg-based Physics, single 2 hours lab):

Simulation vs. Real Equipment

show cck

Simulation testing microcosm of education research

Routinely see examples of principles established invery different contexts.

•cognitive load•construction of understanding•build on prior knowledge•connections to real world•exploration and deep understanding transfer•motivation--factors affecting and connections to learning•perceptions based on organizational structures, structures change and develop, changes perception.•…

What works and why- matches research on learning,tested to ensure it works.

Conclusions:Interactive simulations powerful new technology for learning science. But not automatically good…

Phet.colorado.edu references on website under “research”

1. Think aloud interviews (~200 hours) Explore with guiding question

General results from student interviews*

a. Surprisingly consistent responses, particularly on interface.

b. Vocabulary very serious hindrance to learning anddiscussion-- see because simulation removes

c. Animation attention, but not thinking. Interactivity thinking & learning.

W. K. Adams, et al. , A Study of Educational Simulations Part I - Engagement and Learning., A Study of Educational Simulations Part II - Interface Design.,

• Intuitive interactivity vital• Controls Intuitive when most like hand action

– Grab-able Objects– Click and Drag– Sliders to change numeric values

• Representations– Cartoon-like features scale distortion OK– Good at connecting multiple representations, but

proximity and color coding helps (energy sktprk)

f. A few important interface characteristics*

*more than want to know in Adams et al. papers

Surprising differences in motivation and learning

engaged exploration vs. “performance mode”

Before topic covered in class- actively interested and engaged in figuring out answer

After covering in class (same students, same topic!)struggle to remember what told or read, not use sim to explore and figure out, even with repeated encouragement! Frustrated and unsuccessful!

(ed. research microcosm cont.--matches perfectly with results of Dweck psychology studies- “performance mode” )

many other examples of power of visual models

all of quantum! (S. McKagan)

quantum wave interferencelasersStern-GerlachMRItunneling…

major impact on student thinking

Rethinking how intro quantum mechanics should be taught.Key missing element.

Why these principles revealed so clearly?•cognitive load•construction of understanding•build on prior knowledge•connections to real world/relevance•motivation--factors affecting and connections to learning•…

radical speculation…

Observations of students using and learning from sims.= Better window into student brain-- eliminate language barriers, establish common frame of reference for discussion.

Powerful new tool for education research. Understanding how students best learn topics and hence how to teach.

Why interview results so consistent?

Some general principles revealed for how people learn physics

•Cognitive load important--too much stuff overwhelming,Build up slowly works well. (Radio waves full view)

•Need to connect to prior thinking-- start using familiar elements (tire pump in ideal gas), build understanding

•Visual models vital (balloon and sweater, CCK,QM, …)

•Absorb and make sense only when ask questionsand then seek answers. Animation without interactivity draws attention, but not exploration and understanding.

•Text and explanations usually ignored or misinterpreted unless seeking that particular point.

Visual Models

Research Base

Learning Goals

Initial Design

~Final Design

Redesign

Interviews

Interviews

ClassroomUse

Development Process

Team for each sim faculty content expert(s)sci. ed./user expert(s)software engineer

Initial Design &General Approach

Research base: •Ed. Psych / Cog. Sci: How people learn

•Educational Software Design

•Student Conceptions in Physics

•PhET research findings

Assessment of Design:

• Usability – easy/intuitive• Interpretation – correct/productive• Engaged exploration

• Can students construct understanding of main ideas? Achieve learning goals?

General General Design Design GuidelinesGuidelines

Lecture – Interactive Lecture Demos

Demo 4: Sketch position vs time and velocity vs time graphs for when Moving Man: walks steadily towards the tree for 6 seconds,then stands still for 6 seconds, and then towards the house twice as fast as before for 6 seconds.

+

0

-P

ositi

ontime

Vel

ocity +

0

-

time

5 s 10 s 20 s15 sThorton and Sokoloff, 1997Thorton and Sokoloff, 1997

Vel

ocity +

0

-

time

+

0

-

Pos

ition

time

5 s 10 s 20 s15 s

Vel

ocity +

0

-

time

+

0

-

Pos

ition

time

5 s 10 s 20 s15 s

Vel

ocity +

0

-

time

+

0

-

Pos

ition

time

5 s 10 s 20 s15 s

A B

C

Vel

ocity +

0

-

time

+

0

-P

ositi

ontime

5 s 10 s 20 s15 s

D

Electromagnetic waves: Radio Waves sim

Concept Tests andPeer Instruction

Lecture – Concept tests

The speed of the wave (signal) is measured as…

a.how fast this peak moves to the right. b.how fast this peak moves up and down.c.could be a or b

Incorporating a suite of sims (Modern Physics for Engineers)

Course Pre Postnormalized

gain N

Eng. Sp06 30 65 0.50 156

Eng. Fa05 32 69 0.55 162

Eng. Sp05 (30) 51 0.30 68

Phys. Sp06 44 64 0.36 23

Phys. Fa05 40 53 0.21 54

Phys. Sp05 (44) 63 0.33 64

Quantum Mechanics Conceptual Survey

Reformedwith sims

Traditional

~ 10 interactive simulations in lecture and homework(clickers, real life applications, conceptual homeworks)

Interactive Recitation Study

Reformed large-scale introductory calculus-based physics course with Tutorials

“CCK” (N=180) “Real” (N=185)

vs.

Keller, C.J. et al. Assessing the effectiveness of a computer simulation in conjunction with Tutorials in Introductory Physics in undergraduate physics recitations,", PERC Proceedings 2005

DC Circuit Q's on Midterm Exam

Avg1413128430

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Questions

Fra

ctio

n C

orr

ect

CCK

Real

Interactive Recitation Study

p=0.01

but end of semester exam, no observable difference