Debating the next BIG thing in teaching statistics

Debating the next BIG thing in teaching statistics

Allan Rossman, Beth Chance

Cal Poly – San Luis Obispo

Overview

Goals Stimulate thought and discussion

Five propositions as to what the next BIG thing is About undergraduate, introductory statistics

Set stage for breakout sessions, other plenaries Inspiration

“Nothing tunes the neurons like disagreement.” -- David Moore

Overview (cont.)

Disclaimers: We’re not experts on any of these topics We don’t have sufficient time to do justice to any

of these propositions We’ll give some unsubstantiated opinions We don’t even necessarily agree with some of the

positions we’ll espouse

THE NEXT BIG THING IN TEACHING STATISTICS WILL BE

Removing the letters z and t from introductory courses

Elimination of letters z and t

Not literally! We can’t advertise our discipline as

S_A_IS_ICS We mean the elimination of normal-based (z-

and t-) significance tests and confidence intervals from the introductory course

Motivation

“Ptolemy’s cosmology was needlessly complicated, because he put the earth at the center of his system, instead of putting the sun at the center. Our curriculum is needlessly complicated because we put the normal distribution, as an approximate sampling distribution for the mean, at the center of our curriculum, instead of putting the core logic of inference at the center.”

– George Cobb (TISE, 2007)

Arguments for such a curriculum Randomization model is simple and easily

grasped Randomization model ties data collection

process to inference technique to scope of conclusion

Easily generalizeable to other statistics, other designs

Takes advantage of modern computing Truer to Fisher’s vision of inference

Many have taken up Cobb’s challenge NSF-funded curriculum development projects

Rossman, Chance, Holcomb, Cobb (CSI) West and Woodard Gould et al (UCLA) Garfield, delMas, Zieffler, et al (CATALST)

More have taken up Cobb’s challenge Full implementations

Tintle et al (Hope College) March 2011 JSE article Textbook project

Hamrick et al (Rhodes College) 2011 JSM panel discussion

Lock5 textbook project Tabor and Franklin, Statistical Reasoning in

Sports

BUT …

BUT … Simple and easily grasped?!? Our assessment results have been mixed Many students struggle with reasoning

process even after multiple activities Pre-requisite knowledge?

Model, distribution, “random,” simulation Biggest sticking points

Seeing the big picture of why doing this Realizing/appreciating that simulation assumes null

model to be true Understanding why look beyond observed result

Granted …

Student performance may improve with full integration throughout curriculum, complete materials/textbook

BUT… This has been tried before … Wardrop, Statistics: Learning in the Presence

of Variation (1994) Simulation based Early exposure to inference Normal based methods don’t appear until last 1/3

This approach did not catch on Ahead of its time? Not viable for publishers?

BUT…

Students still want to learn z- and t-procedures Many find comfort, familiarity in the (apparent)

exactness of normal probability calculations Students still need to learn z- and t-

procedures Those procedures still dominate statistical

practice in other fields And will continue to do so?

Although…

Randomization methods are become more widely used and accepted not only in statistics but also in client disciplines Manly, Randomization, Bootstrap, and Monte

Carlo Methods in Biology, 3rd ed., 2006

More discussion: Randomization curriculum Breakout sessions

11am today (panel discussion on implementation) 3pm today (Lock and Lock: bootstrapping and

randomization) 11am tomorrow (Lock, Lock, and Lock:

technology demonstrations) Technology demo

4:30pm today (West, StatCrunch)


Students entering introductory college courses with considerable knowledge of statistics

Students will know lots of statistics Common Core State Standards Initiative

State-led effort coordinated by National Governors Association and Council of Chief State School Officers, released 6/2/2010

Standards define the knowledge and skills students should have within their K-12 education careers

Currently adopted by 42 states Two assessment consortia (testing in 2014-15) www.corestandards.org

Common Core – Mathematical Practice Standards Foster reasoning and sense-making in

mathematics Reason abstractly and quantitatively Construct viable arguments and critique the reasoning of others Model with mathematics Use appropriate tools strategically [technology]

Common Core – Statistical Concepts 6th grade:

Develop understanding of statistical variability Summarize and describe distributions

7th grade: Investigate chance processes and develop,

use, and evaluate probability models High school:

Using probability to make decisions Making inferences and justifying conclusions

Can you imagine students who? Have already mastered

Variability Distribution Sampling, Experimentation Statistical Inference

Have been consistently asked to Critique Reason Model Use technology

Jerry Moreno’s perfect world

“In 7 years or so, STATS 101 has been revised so to excite the CC student by: Beginning the course with several real world

projects/case studies that review/address/ challenge the content and mathematical practice base of CC statistically literate students;

Continuing the course with topics such as: Normal theory inference; risk analysis; design of experiments/clinical trials; anova;….”

-- CAUSE webinar, May 2011

What could we do with such students? Mean vs. median? Risk analysis (e.g., Utts, 2010)

Multivariate modeling (e.g., Kaplan, 2009)

Large, complex data sets, data mining (e.g., Gould plenary talk)

Bayesian methods, decision theory (e.g., Stewart plenary talk)

Computing, visualization tools (e.g., Nolan and Lang, 2010)

Data dialogues (e.g., Pfannkuch et al, 2010)

Essential (and cool!) skills …

“I keep saying that the sexy job in the next 10 years will be statisticians. And I’m not kidding. Now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it. -- Hal Varian, Chief Economist, Google

BUT …

BUT … Alternative standards

Design and conduct statistical experiment, interpret and communicate outcomes

Construct and draw inferences from graphs Understand and apply measures of center,

variability, association Use curve fitting for predictions Apply transformations of data

BUT … Alternative standards (cont.) Understand sampling and recognize its role

in statistical claims Use simulation to estimate probabilities Create and interpret discrete probability

distributions Use properties of normal curve to answer

questions about relevant data

BUT … What’s the point?

These alternative standards are more modest than Common Core Perhaps more realistic to attain? But could still require a fundamental change in

content of introductory college courses 1989 NCTM Curriculum and Evaluation

Standards for School Mathematics Have we substantially changed content of Stat

101 in past 22 years based on students’ achieving these standards?

Granted…

Common Core has a lot more political might, buy-in from important stakeholders Much higher probability of impact

BUT … Another big concern

Preparing current and future teachers to implement such a curriculum is a big challenge Need considerable professional development for

current teachers Need to substantially re-think teacher preparation

for prospective teachers

More discussion: Common Core Breakouts

11am today (Starnes: AP Stats, Nspire CX, and Common Core)

11am tomorrow (Scheaffer and Franklin: K-16 Common Core)


The disappearance of print textbooks

Let’s acknowledge

Students don’t read textbooks See textbooks as a (very expensive!) repository of

homework problems Perhaps also skim examples hoping to mimic for

homework problems Students don’t keep textbooks as reference

Today’s students are “digital natives” Very comfortable looking to internet, Wikipedia as

reference

Example data

Students more highly value instructors’ notes, instructor-driven decisions How useful did you find the following learning

aids/materials in helping you understand statistics? (77-78 responses)1 = Not helpful, 5 = Most helpful, skip the question if you did not use the resource consistently

More importantly

Print textbooks aren’t dynamic enough to support learning Can’t evaluate a student response and

provide guiding comments Not conducive to allowing students to work

non-linearly Can’t easily jump around to what they need

Examples can become outdated very quickly Can’t adapt to student interests on the fly

Instead?

Integration of hot-off-the-press case studies Adaptable presentation Interactive demonstrations Optional drill and practice Immediate individualized feedback Flexibility in timing and presentation Replayable podcasts Interactive online surveys

Some examples

ActivStats, CyberStats, SOCR, HyperStat Carnegie Mellon’s Open Learning Initiative The Open University (U.K.) Publisher learning systems

StatsPortal (Exhibitor Test-Drive), WileyPlus, …

BUT …

What technology innovation has had the greatest impact on education? Printing press!

BUT …

Books have had huge impact on education Textbooks maintain firm hold on U.S. higher

education College faculty members (as a group) are very

resistant to change Some of these multimedia materials have been around for

a while and have not taken over the world Even if the use of print textbooks lessens

considerably in the next few years … Print textbooks are not going away!

Compromise?

What’s needed is access to plethora of resources for instructor/student to pick and choose from

Not one (extra large) size (print textbook) fits all

And then Server-side database maintaining individualized

interactive student texts Add notes to eBook in class Submission of work for instructor-embedded feedback


Online and hybrid courses replacing face-to-face interactions among students/students and instructor

No more face-to-face classes

With all of these multimedia materials, why do we require students to Sit in (uncomfortable) seats At the same place at the same time Often without access to any resources beyond

paper and pencil? Why not let students work at their own pace,

using technology, when it’s convenient? Students at Cal Poly typically avoid Friday classes

More interaction?

Some students interact better online, overcome reluctance to participate in person

On-line office hours, whiteboards e.g., elluminate

Calibrated-peer-review model

Growing popularity and importance Class Differences: Online Education in the United

States 2010 (Sloan Consortium) 63% of reporting institutions said online learning was a

critical part of their long term strategy, compared to 59% in 2009

Nearly 30% of U.S. higher education students took at least one online course in 2009, compared to 20% in 2006, 10% in 2002

Many more institutions reported seeing an increase in demand for online courses and programs than for face-to-face.

Economics!

Online courses do not compete for scarce classroom space

“Across the country, traditional colleges are struggling, but for-profit schools such as the University of Phoenix are experiencing tremendous growth.” Moneywatch (2010) 438,000 students in 2010 Largest private university in U.S.

Comparison of student performance “On average, students in online learning

conditions performed better than those receiving face-to-face instruction.” Evaluation of Evidence-Based Practices in Online

Learning: A Meta-Analysis and Review of Online Learning Studies, U.S. Department of Education, September 2010

BUT

BUT 50 years ago …

Another exciting new technological marvel was predicted to replace face-to-face class meetings between instructor and students

Frederick Mosteller pioneered the teaching of statistics via …

TELEVISION!

BUT 50 years ago…

“In the early and mid 1960s, television was the great technological hope. Here is a quote from Time magazine: ‘Not only is a taped professor as informative as a live one, but he seldom turns sour and never grows weary of talking.’ There was actually a feeling that taped teaching by master teachers would replace live teachers on campus as well as taking advantage of the reach of broadcast television.” -- David Moore (1993)

BUT 50 years ago…

“It's very likely that a course taught on television, because of the careful preparation, will be better organized lecture by lecture than the usual lecture in class, but it does have a lack of flexibility…. The idea that certain materials can be expressed better in a tv session seemed to me to be right, and still can be right. I think that the expanded ability to produce material that has more visual content than anything we were able to put together adds a lot more interest to the course.” -- Fred Mosteller (1993)

Granted …

Online courses have great potential for interactivity that televised courses do not

But in some (many?) online courses the instructor merely delivers information passively to students

BUT …

“Social interaction plays a fundamental role in the process of cognitive development” (Vygotsky) Granted, today’s students are very comfortable

with socializing online But our sense, and our own experience, is that

(synchronous) face-to-face discussions can be much more efficient and productive than working (asynchronously) online

Is there something special about face-to-face social interaction with regard to learning?

Compromise?

Different model for face-to-face classes Students complete background reading/ podcast

with guided questions, drill and practice prior to attending class (literacy)

Class time is spent working examples, presenting solutions, asking questions (of other students and instructor), teamwork, peer instruction

Examples “Inverted Classroom” (e.g., Mazur; Lage, Platt, Treglia) “Statistical Reasoning Learning Environment” (e.g.,

Garfield & Ben-Zvi, 2008)

More Discussion: Online Teaching Breakouts

11am today (Fairborn and Zeitler: Transition to Online Teaching)

11am tomorrow (Everson and Miller: Social Media)


Curriculum and pedagogy decisions will be grounded in educational research

Statistics Education Research May still be in its infancy as a discipline

But has enjoyed a tremendous growth spurt! Journal of Statistics Education

Founded at N.C. State in 1993 Nearing its 20th anniversary Publishing high-quality, rigorously refereed

scholarship Including more and more research articles

More statistics education research Statistics Education Research Journal

Nearing its 10th anniversary (launched 2002) Publishing exclusively research articles in statistics

education Ph.D. Dissertations

IASE website lists 70 Ph.D. dissertations in statistics education since 2000 Including many from researchers here today Probably many more not listed there

U of Minnesota Ph.D. program in Statistics Education (8 students in fall)

Ph.D. program to be developed at U of Georgia

More statistics education research Models of Qualitative and Quantitative

methods Using statistics effectively in mathematics

education research (ASA, 2007) SRTL Research forums SERJ special issue (Nov, 2010) Second Handbook of Research on Mathematics

Teaching and Learning (Lester, 2007)

More statistics education research CAUSE

Research Advisory Board Led by Joan Garfield since inception of CAUSE

Research Clusters 2007-09: 3 clusters with 11 participants 2009-11: 3 clusters with 12 participants Grant proposals, journal articles and presentations at

national and international conferences

Connecting Research to Practice? JSE has a new feature titled “From Research

to Practice”

Garfield and Ben-Zvi, Developing Students’ Statistical Reasoning: Connecting Research to Practice, Springer, 2008.

Example – The Statistics Pathway (Carnegie Foundation, Dana Center) Development of one-year curriculum in

statistics, data analysis and quantitative reasoning for developmental math students equivalent to one-semester college course

Collaboration of representatives of several professional organizations, statistics educators (2 and 4 year), developmental mathematics educators (2 year), researchers, and designers, access to policy makers

Design of Statway curriculum, materials, teaching routines is evidence-driven Based on hypotheses grounded in ed, math ed

and stat ed research, practitioner experience Hypotheses tested and refined as Statway is

implemented by community college faculty Revisions guided by evidence of student

learning, experiences of faculty implementers Eliciting diverse sources of expertise Building on open source materials

BUT…

Statistics education research can provide sound principles, but think about how many decisions instructors make on a daily basis

Example: Statistical significance for 2×2 tables

Learning Goals: Understand concept Apply relevant procedure to real data Interpret results Draw appropriate scope of conclusions Explain impact of various factors such as group sizes

BUT I have to decide…

Which method to present first? Which to present at all? Simulate randomization test, Fisher’s exact

test, Two-proportion z-test, Chi-square test Describe method first, or try to ask questions

to lead students to suggest method? Present example through lecture, or guided

activity, or on-their-own activity or …?

OK, simulation. So now have to decide: Start with tactile simulation or technology?

Which technology to use? Should students design own simulations or press

buttons? Choice of dataset

Real or realistic? Randomized experiment or independent random samples or neither? Significant difference or non-significant?

Choice of test statistic Difference in success proportions or number of

successes in group A or relative risk or odds ratio or …?

Still more decisions

How many examples to present? With what characteristics?

How to assess student learning to guide learning? Group quiz, individual quiz, homework

assignment, mini-project, multiple choice questions, …?

BUT …

Not many research studies in statistics education compare several options and try to identify the most effective With sufficient replication for results to be

generalizable Not feasible to ask for research studies in

such a young field to address all of these small decisions Decisions instructors make every single day

BUT … Another big hurdle

College faculty members as a group are very resistant to change Yes, we’ve said this before

College faculty members as a group do not like to be told what to do Even when that advice is based on rigorous

educational research College faculty members are often skeptical

of education research Especially qualitative research

Compromise?

Research can continue to establish general principles For example, active is better than passive learning

Instructors can be trained to use their judgment on how to apply them in their particular setting And given the freedom to do so

Develop and support more teacher-scholars in statistics education

More Discussion: Statistics Education Research Statway: Kristen Bishop, Dana Center Plenary: Bob delMas Breakouts:

11am today (Zieffler & Mvududu: Qualitative methods)

3pm today (Lovett: Qualitative data) 11am tomorrow (Hilton and Enders: Conceptual

framework)

Let’s Review

Eliminating z and t has potential But not a magic bullet

Future students will know more statistics before college So we need to get prepared

Textbooks aren’t going away But instructors need better access to plethora of

open-source, collaborative resources

Let’s Review

Online learning, multimedia resources will continue to gain in popularity & accessibility Opportunity to change classroom experience

Research can lead to more effective curriculum and pedagogy Needs to be closely tied to teaching practice

For more debate Breakout 11am today (Peck)

So…

Focus more logic of inference Students will come in knowing statistics Textbooks need to change Have more interactive class sessions Learn from the research

Many of these ideas are not so new…

Why BIG now and not before? Improved technology and understanding of

how to use technology for good More availability and appreciation of data Students are changing Better understanding of student learning

Including specific to statistics More buy in, alignment of stars More insights: Pearl dinner presentation

Take Home Message

Engage students

Persist in face of resistance

Break shackles

Enjoy the conference!

Thanks very much!

[email protected] [email protected]

Documents

Debating the next BIG thing in teaching statistics