SACS Conference Nashville - Authentic Educationalso required use of the Pythagorean theorem. ‘It seems applying the Pythagorean theorem was a weakness for kids,’ said William Kendall,

Understanding By Design Fall 2003

© 2003 Grant Wiggins

Page ‹#›

© 2003 Grant Wiggins & Jay McTighe UBD 10/20031

Results - by DesignSACS Conference

NashvilleDecember 2003

Grant Wiggins© 2003 Grant Wiggins & Jay McTighe UBD 10/20032

Introductory Q’sKey Questions –

n How can we promote understanding and highlevels of achievement more by design thanby good fortune (and native ability)?

n What is the ideal relationship between localand state assessment?

n How do we move beyond designing mereinteresting activities or textbook “coverage”?

n How does common sense inform our work?When is “common sense” mistaken?


3 Understandings:n The goal is transfer, the ability to use your

abilities wisely, on your own. Life is notmultiple choice but authentic assessment.

n Research shows that most educatorsmisunderstand state testing: they incorrectlyassume that teaching to the test is the best wayto maximize results. No, the need is to teach tothe standards with high-quality localassessment

n It’s not “teaching” that causes learning. Onlyattempts by the learner to learn causeunderstanding. That requires less “teaching”and more “coaching” on worthy tasks © 2003 Grant Wiggins & Jay McTighe UBD 10/20034

Key need in designOvercoming the TWIN SINS:

“Aimless Activity” and“Superficial Coverage”

n A better focus on goals, related tobig ideas and complex performance

n Moves us beyond overly-fragmentedand isolated lessons & activities


worth being familiar with

important toknow & do

Big ideas& core tasks

Establishing Intellectual prioritiesaround “Big Ideas” & “Core Tasks”

“big ideas”& core tasks worth

exploringin depth

foundational knowledge & skill

”nice to know”


Some questions for identifyingtruly “big ideas”

n Does it have many layers and nuances, notobvious to the naïve or inexperienced person?

n Can it yield great depth and breadth of insightinto the subject? Can it be used throughout K-12?

n Do you have to dig deep to really understand itssubtle meanings and implications even ifanyone can have a surface grasp of it?

n Is it (therefore) prone to misunderstanding aswell as disagreement?

n Are you likely to change your mind about itsmeaning and importance over a lifetime?

n Does it reflect the core ideas in a field or in life,as judged by experts?



Page ‹#›


Also: Identify Core TasksWhat core tasks do the discrete skills and

facts point toward & enable?n Core tasks provide answers to: “Why are we learning

this? What does it help you do?”n Think of core tasks as the decathlon in your field -

mastery of the complex challenges faced bypractitioners routinely, in each discipline

n Core tasks require TRANSFER - responding, on yourown, with a repertoire and good judgment - i.e.understanding - to challenges

n requires using content wisely, with minimal cues andprompts: different from mere ‘plugging in’ orformulaic responses


Key Analogy, then: Coaching“What core complex & worthy tasks

do the standards imply? Whatmust students be coached to do

well, on their own?”

Think through –n Assessment of their ability to

play “the game”not just respond tothe “drills” (test items)


From the Boston Globe articleon the MCAS math scores

‰ “The hardest question on the math section, which just 33percent got right, asked students to calculate the distancebetween two points. It was a cinch -- if students knew thatthey could plot the points and use the Pythagorean theorem, awell-known formula to calculate the hypotenuse of a righttriangle if the lengths of two legs are given. The sixth-hardestmath question, which only 41 percent of students got right,also required use of the Pythagorean theorem. ‘It seemsapplying the Pythagorean theorem was a weakness for kids,’said William Kendall, director of math for the Braintreepublic schools. ‘These weren't straightforward Pythagoreantheorem questions. They had to do a little bit more.’”

n page B1 of the Boston Globe on 8/4/2003.


NAEP 12th grade item, sameconcept

n What is the distance between thepoints (2,10) and (-4, 2) in the xyplane?

‰6‰8‰10‰14‰18

nCorrect responses: 32%


Another NAEP item onPythagorean Theorem

30 in

ches

40 inches

What is thediagonal

measurement ofthe TV screen?

n 25n 35n 50n 70n 1200

8th-grade: 25% correct 12th-grade: 42% correct© 2003 Grant Wiggins & Jay McTighe UBD 10/200312

Test Q: NY - same concept30 To get from his high school tohis home, Jamal travels 5.0 mileseast and then 4.0 miles north. WhenSheila goes to her home from thesame highschool, she travels 8.0 miles eastand 2.0 miles south. What is themeasure of the shortest distance, tothe nearest tenth of a mile, betweenJamal’s home and Sheila’s home?[The use of the accompanying gridis optional.]

63% incorrect



Page ‹#›


Another item - gotten wrong by3/4 of NY 11th-graders

34 A straw is placed into arectangular box that is 3inches by 4 inches by 8inches, as shown in theaccompanying diagram. Ifthe straw fits exactly intothe box diagonally fromthe bottom left frontcorner to the top rightback corner, how long isthe straw, to the nearesttenth of an inch?


NY Results and a comment...‰ A preliminary survey of school districts by the

Education Department in June found that only about 37percent of the students who took the 35-question testpassed.

‰ ''It used to be with the old Math 1, 2 and 3 tests that youcould just bone up on old tests and pass them and nothave a firm grasp of the concepts,'' Mr. Hazen said.''But now you are looking at a standards-basedassessment, and it is very difficult to look at the pasttests and know what will be on the next test.'' said BobHazen, president of the Association of MathematicsTeachers of New York State.


MCAS test item: 10th-gradeEnglish reading/response item

A fellow fourth grader broke the news to me after she saw myeffort on a class assignment involving scissors and construction paper.“You cut out a purple bluebird,” she said. There was no reproach in hervoice, just a certain puzzlement. Her observation opened my eyes—not that my eyes particularly help—to the fact that I am colorblind. Inthe 36 years since, I’ve been trying to understand what that means. I’mstill not sure I do….

Unlike left-handers, however, we seem disinclinedto rally round our deviation from the norm. Thus there’s no readysource of information about how many presidents, or military heroes,or rock singers have been colorblind. Based on the law of averages,though, there must have been some. We are everywhere, tryingto cope, trying to blend in. Usually we succeed. Until someone spotsour purple bluebirds. Then the jig is up.


The most wrong item on thetest: 70% incorrect:

n This selection is best described as‰ A. a biography.‰ B. a scientific article.‰ C. an essay.‰ D. an investigative report.

n Many students said it could not bean essay because “it was funny”and because “it had more than 5paragraphs.”


Key, then, to local reform:The design and use of powerful

local assessments, requiringtransfer of learning around big

ideasn Must involve high-level application, not

plugging in or mere response to specificcue

n the use of big ideas and core contentn justifying your response - showing your

work© 2003 Grant Wiggins & Jay McTighe UBD 10/200318

Core tasks are key authentic“doings” of the subject

“Worthy tasks, in realistic contexts”nAsk:

‰What realistic work, in what messysituations, do expertsand citizens actually face?

‰What does it mean todo history, mathematics, science,etc.?



Page ‹#›


Task Example #1n “Hoops” McGinty wants to donate millions of dollars

from his salary and sports-drink earnings toward aspecial exhibit in the new Rose Planetarium area of theMuseum of Natural History. Hoops wants there to be a 3-D scale model of our planetary system. There is a catch,however. The size of the planets and the distance ofeach planet from the sun must be exactly to scale –where the sun is represented by a regulation NBAbasketball. The nervous folks in the gifts department ofthe Museum call you up to their office because of yourexpertise in astronomy.

n What will you advise them about the feasibility of theplan? What approach toward a scale model will workbest to ensure a basketball-related design?


Example #2 - A. P. US Historyn Your goal is to determine why the urban riots of the

late 60's happened. You are one of many augustmembers of an LBJ appointed panel, the KernerCommission, who must report to the president andthe country on why the violence happened and whatcan be done about it.

n You will produce a collective report that must bethoughtful, thorough, and clearly presented. Yourpersonal contribution will be judged through journalentries, observations of work and discussion, andsections of writing you produce.


Agree on “core tasks” perprogram (via Standards)

Example: Mathn Make sense of messy – inconclusive,

incomplete, misleading – datan Model a real-world phenomenonn Solve multi-step non-routine problemsn Choose technology wisely and use it to

solve complex problems (spreadsheets,graphing calculators, etc.)


“core tasks”Example: History

n Makes sense of multiple, conflicting primary &secondary accounts, construct your bestnarrative

n Critically research the merit and accuracy ofcited sources, e.g. on the Internet

n Journalistic background: do a helpful history ofa present problem/issue/event = e.g. 9/11

n Design informative multi-media historicalexhibits to enlighten about causes/effects


“core tasks”Example: Science

n Design and de-bug a worthy experiment fromscratch

n Critically evaluate the research of others - peerreview

n Show evidence of having carefully considered theethics and public policy issues of scientificresearch

n Adapt scientific research to practicalsolutions/inventions

n Propose and design tests of a novel hypothesis© 2003 Grant Wiggins & Jay McTighe UBD 10/200324

Understandingunderstanding

Understanding is –n insightful and fluent use of

knowledge and skill, manifest ineffective work, at worthy tasks

‰ cf. Bloom’s “application”‰ revealed via the “six facets”‰“ transfer-ability” is KEY

(as opposed to only cuedresponses to familiar exercises)



Page ‹#›


6 Facets of Understanding:key abilities

n explain – give theories - ‘the why’n interpret – see meaning, stories,

translations made by studentn apply –knowledge in a (new) concrete

contextn take perspective – awareness of other

points of view, critical stancen be empathetic – “walk in the shoes of...”n show self-knowledge – wisdom, “knowing

thyself”, aware of one’s prejudices andhabits of mind


1. Identify desired results

2. Determine acceptable evidence

3. Plan learning experiences & instruction

3 Stages of(“Backward”) Design


Identify content

Brainstorm activities

Come up with an assessment andlink it to some Standard

Typical Error in Design

Without checking for alignment

Without checking for alignment


Overarching understandings

Knowledge and skill to be acquired

Essential Questions

Understanding – by Designn Transfern Rethink, via

big ideasn 6 facets key to

assessmentdesign

‰ Explain‰ Interpret‰ Apply‰ perspective‰ Empathy‰ Self-knowledge

n 3 Stages ofBackward Design

‰ Built upon the desiredlearnings, not thecontent or teachermoves

‰ Focused on big ideas‰ All 3 stages align

n Anticipatemisunderstandings

n Grounded inperformanceassessment


Backward Design logicn STAGE 1: If the desired results are for

learners to...‰ Understand that...‰ Be able to handle such challenges as...

n STAGE 2: then, you need evidence of thestudent’s ability to...

‰ [General evidence needs, regardless of taskspecifics]

n STAGE 3: so, the learning activities mustinvolve...





3 Stages of Design



Page ‹#›


Pointing to big ideas viaessential questions“How does one lead children todiscover the powers and pleasures [ofrethinking]? Through organizingquestions. They serve two functions:they put perspective back in theparticulars... and they often served ascriteria for determining where studentswere getting, how well they wereunderstanding, whether anything newwas emerging.” – Jerome Bruner

Q


Essential QuestionsWhat questions –

n are arguable - and important to argue about?n are at the heart of the subject?n recur - and should recur?n raise more questions – provoking and

sustaining engaged inquiry?n often raise important conceptual or strategic

issues?n can provide organizing purpose for

meaningful & connected learning?

Q


Sample Essential Questions:n Who are my true friends - and how do I

know for sure?n How “rational” is the stock market?n Does a good read differ from a ‘great

book’? Why are some books fads, andothers classics?

n To what extent is geography destiny?n Should an axiom be obvious?n How different is a scientific theory from a

plausible belief?n What is the government’s proper role?

Q


Misconceptions as anotherway to identify big ideas“[What] an extensive research literaturenow documents is that an ordinary degreeof understanding is routinely missing inmany, perhaps most students. If, when thecircumstances of testing are slightlyaltered, the sought-after competence canno longer be documented, thenunderstanding - in any reasonable sense ofthe term - has simply not been achieved.”

– Howard Gardner, The Unschooled Mind


E.g. Misconceptions inscience

From 2061 Benchmarks (AAAS):“ Some students think that ‘cold’ is being

transferred from a colder to warmerobject…students often think thatobjects cool down or release heatspontaneously…

Even after instruction, students don’t alwaysgive up their naive notion that some

substances (e.g. flour) cannot heat up, or thatmetals get hot because they “attract heat”

etc.” (pp. 337-8)© 2003 Grant Wiggins & Jay McTighe UBD 10/200336

Knowledge vs. Understandingn An understanding is an unobvious and

important inference, needing “uncoverage” inthe unit; knowledge is a set of established“facts”.

n Understandings make sense of facts, skills,and ideas: they tell us what our knowledgemeans; they ‘connect the dots’

n Any understandings are inherently fallible“theories”; knowledge consists of theaccepted “facts” upon which a “theory” isbased and the “facts” which a “theory”yields.



Page ‹#›





3 Stages of Design:Stage 2


Reliability: Snapshot vs.Photo Album

We need patterns thatovercome inherentmeasurement error

n Sound assessment (particularly of StateStandards) requires multiple evidenceover time - a photo album vs. a singlesnapshot


Other EvidenceWe supplement “performance”

evidence with targeted test itemsn Need both kinds, but we must

subordinate quizzes to tasks, as insports drills vs. the real game

n Construct parallel versions of tasksand quizzes to increase reliability


2 Questions for apractical test of your tests:

1. Could your test be passedwithout in-depth understanding?

2. Could the specific test result bepoor, but the student stillunderstand or be able toeffectively apply the ideas inquestion?

The goal is to answer NO to both

!


“Backward Design” of assessmentfrom state standards:

What evidence is implied in the standard?

What evidence is implied in the standard?

What is the standard? What is the standard?

What kinds of “tests” will provide such evidence?

What kinds of “tests” will provide such evidence?

What should we look for (score via rubrics), given the desired result?

What should we look for (score via rubrics), given the desired result?


Implications: Agree on corerubrics and core tasks

Key rubrics as well as tasks are explicitand implied in the standards

n rubrics for charting progress against keyperformance traits - critical thinking,effective problem-solving, reading andwriting fluently in key genres, etc.

n Key tasks as the most importantperformances requiring such work:defensible research, multi-step problems,speak and write fluently, etc.



Page ‹#›


Don’t confuse the drills with thegame (authentic performance)

‘Drill-tests’n Out of contextn Discrete, isolated

elementn Unrealistically set

up and promptedn Doesn’t transfer

without practice ofthe game itself

The ‘game’n In context, with all

its messiness andinterest value

n Requires arepertoire, usedwisely

n Not prompted: youmust judge what todo, when; you mustadapt prior learning


Authenticity of core tasksn As defined by Newmann et al.:

‰ construction of knowledge‰ disciplined inquiry‰ value beyond school

n "A Guide to Authentic Instruction and Assessment”

n As defined in UbD:‰ realistic task, context, standard‰ requires judgment and adjustment, not plugging in‰ minimal secrecy and arbitrary constraint on resources,

assistance


Reform mantra

Assess what we value,and value what we assess

i.e. Move beyond:n ”Test what is merely easy to test and

grade"n "Teach, test, and hope for the best"


Don’t confuse cause andeffect in testing!too many educators focus on the

annual ‘audit’ instead of dailyperformance!

n Teaching to the test only is likepracticing the physical all year long

‰ State tests audit “fitness” indirectly, so localassessment must not mimic the audit

n The quality of local assessmentdetermines the results on the audit


Ponder...

nNo, it’s not “practice makesperfect”...

nOnly perfect practice makesperfect

nEspecially since practicemakes permanent


A Body of Evidence requiresauthentic assessment

Look at the Standards:The verbs signify key complex

capacities, not just discrete skillsn The local assessment system must be anchored

in core performances that embody theStandards

n Multiple tasks, over time, must be usedn Other, more traditional evidence for rounding

out the picture



Page ‹#›


Policy must ensure that localassessment is excellent

Local assessment needs models,design standards, and criteria

nLocal assessment must focuson understanding; and on fluentperformance in complex, worthytasks


Needed: High-Quality LocalAssessment

“Local control with quality control”:n A balance between:

‰ classroom‰ external audits

n Local assessment and grading standardsneeded

n Peer review of assessment design and peerscoring of student work are key to buildinglocal capacity & excellence

n District-wide rubrics based on exemplarssend a clear and consistent message toeveryone


Standardized TestsDeliberately Audit

n Don't confuse validity with authenticity‰ e.g.: SAT2 is the best predictor of G.P.A. at Williams

College‰ e.g. cloze method is valid (though inauthentic) measure of

reading for comprehensionn The SAT1 is a measure of general analytic

ability, not linked to curricula by design‰ Margin of error: 32 points either way‰ Course difficulty cited as key statistic against which to

compare test scores


Research - Learning andAssessmentn Newmann et al. (1996) measured how well 24

restructured schools implemented authenticpedagogy and authentic academic performanceapproaches in mathematics and social studies.

n Students with high levels of authentic pedagogy andperformance were helped substantially whether theywere high- or low-achieving students. Anothersignificant finding was that the inequalities betweenhigh- and low-performing students were greatlydecreased when normally low- performing studentsused authentic pedagogy and performance strategiesand assessments.


Chicago researchn Assignments were rated according to the degree to

which they required “authentic” intellectual work:“Students who received assignments requiringmore challenging intellectual work alsoachieved greater than average gains on the IowaTests of Basic Skills in reading andmathematics, and demonstrated higherperformance in reading, mathematics, andwriting on the Illinois Goals AssessmentProgram...


Chicago, cont.n “Contrary to some expectations, we found high-

quality assignments in some very disadvantagedChicago classrooms and [found] that allstudents in these classes benefited fromexposure to such instruction. We conclude,therefore, [that] assignments calling for moreauthentic intellectual work actually improvestudent scores on conventional tests. (p. 29)

The complete research is available online athttp://www.consortiumchicago.org/publications



Page ‹#›


Authentic work, defined in thestudy:

n “Authentic intellectual work involves original applicationof knowledge and skills, rather than just routine use offacts and procedures. It also entails disciplined inquiryinto the details of a particular problem and results in aproduct or presentation that has meaning or valuebeyond success in school. We summarize thesedistinctive characteristics of authentic intellectual workas construction of knowledge, through the use ofdisciplined inquiry, to produce discourse, products, orperformances that have value beyond school. (pp. 14-15)


Research - Learning andCognition

n “Learning with understandingis more likely to promotetransfer than simplymemorizing information from atext or a lecture.” - Bransford, et. al., How People Learn, p.

224


feedback and its use is key togreat gains

Black & Wiliam meta-analysis:“There is a body of firm evidence thatformative assessment is essential... Weknow of no other way of raisingstandards for which such a strong primafacie case can be made.”

n Black and Wiliam (1998) “Inside the BlackBox: Raising Standards through ClassroomAssessment,” Phi Delta Kappan, volume 80,

2 (October), pp. 139 ff.© 2003 Grant Wiggins & Jay McTighe UBD 10/200358

Feedback: the element inHarvard’s “most effective”courses

from the Harvard AssessmentSeminars:

n "The big point—it comes up over and over again ascrucial—is the importance of quick and detailedfeedback. Students overwhelmingly report that thesingle most important ingredient for making a courseeffective is getting rapid response on assignments andquizzes.

n "Students suggest that it should be possible in certaincourses to get immediate feedback. They suggest thatthe professor should hand out an example of anexcellent answer.


Feedback as key toeffectiveness (cont.)

Harvard Assessment Seminars:n "Secondly... an overwhelming majority are

convinced that their best learning takesplace when they have a chance to submit anearly version of their work, get detailedfeedback and criticism, and then hand in afinal revised version...

n Many students observe that their mostmemorable learning experiences have comefrom courses where such opportunities areroutine policy."


Ironically, also true forteachers

“Faculty members at Harvard were askedwhat single change most improved theirteaching. Two ideas swamped all others.One is enhancing student awareness of thebig picture, ‘the big point of it all’. Thesecond is the importance of helpful andregular feedback from students so aprofessor can make midcourse corrections.”

Harvard Assessment Seminar



Page ‹#›


Stage 3 Design Standard

EFFECTIVE

ENGAGING

and


Design mindful of “bestdesigns”Build lessons mindful of the implications

of the “Best Design” exercise:n What was the best-designed experience you ever

were in, as a learner? What made it so effective andengaging?

n Given the various examples, what are the commoncharacteristics? What are the transferable lessonsabout good design?


Most common answers K-16:n Clear goals and explicit performance requirementsn Models and modeling providedn A genuine challenge/problem that stretches you - real,

meaningful work, hands-on learningn Genuine application required in assessmentn Trial and error, reflection and adjustment are expected,

encouraged and designed inn Constant helpful feedbackn The teacher is more of a facilitator, coachn There is a safe, supportive environment for risk-takingn Variety, choice, and attention to difference anticipatedn A good mix of collaboration/solo workn Immersion, active, multi-sensory experience © 2003 Grant Wiggins & Jay McTighe UBD 10/200364

Organize byW. H. E. R. E. T. O.

Where are we headed? (the student’s Q!) How will the student be ‘hooked’? What opportunities will there be to beequipped, experienced, and explore keyideas? What will provide opportunities to rethink,rehearse, refine and revise?How will students evaluate their work?

How will the work be tailored to individualneeds, interests, styles?

How will the work be organized for maximalengagement and effectiveness?

WHE

ER

L

TO

Overarching EssentialQuestion(s)

OverarchingUnderstanding(s)

unit 1unit 2

unit 3unit 4

unit 5

unit 1unit 2

unit 3unit 4

unit 5

unit 1unit 2

unit 3unit 4

unit 5

unit 1unit 2

unit 3unit 4

unit 5

Course 1 Course 3 Course 4Course 2

CoreTask(s)

System-Wide Rubric(s)


Whose “story” is it?

Who is an American? Who says?

Nesting Overarching EssentialQuestions

Is “all fair” in war

(internment)?

WW II

Who should get

Green cards?

80’s60’s

How muchdoes racematter?



Page ‹#›


Subject: How does an author useirony to make us understand?

Nesting Overarching EssentialQuestions: English

Course: Who sees? Who doesn’t?Why?

Why doesn’t Oedipus get it?

OedipusWhy can’t Pooh

see he isn’t tracking Woozles?

PoohPlatoHow doessomeoneever leavethe Cave?


Program: What’s the Pattern?How do you Know

Nesting Overarching EssentialQuestions: Mathematics

Subject: What is the “best fit”?

Is this a linear relationship?

linear

What is thetessellation

here?

Transforms.non-linear

Is this anexponential

relationship?


Overarching (Course)Understandings

Overarching Understandings highlight therecurring & transferable lessons in a

course, subject, and programn English: “Constant reflection on audience

and purpose is key to effective writing andspeaking”

n Math: “Much of math involves a modelingcycle:

‰ using abstractions to represent things,‰ manipulating the abstractions via logical rules,‰ checking how well results match the original thing”

(from AAAS Science Literacy Atlas)© 2003 Grant Wiggins & Jay McTighe UBD 10/200370

Boyer’s Basic School - “CoreCommonality” & Questions “Everyone holds membership in a

variety of groups”:

n Which groups did I join at birth?n Which groups do I belong to?n Why do people join groups?n Can I leave a group?

Q


for further information...Contact us:

nE-mail: [email protected]: www.grantwiggins.org

Documents

SACS Conference Nashville - Authentic Educationalso required use of the Pythagorean theorem. ‘It seems applying the Pythagorean theorem was a weakness for kids,’ said William Kendall,