6
1 What is Data Literacy? © 2011 Using Multiple Sources of Data DATA LITERACY for Teachers by Nancy Love There is little dispute that educators’ effective use of school data is a hallmark of improving schools. But all the data in the world will have little impact on student achievement unless teachers feel comfortable, knowledgeable, and skilled in using a variety of data on a regular basis to improve teach- ing and learning. This reference guide provides a simple framework for strengthening data literacy and answering these questions: • How can schools move away from over-reliance on state assessments and from using data as either a carrot or a stick? • What kinds of data do teachers analyze, individually and with colleagues? For what purposes? How often? • How can data use have an immediate and direct impact on instruction and achievement? Data literacy is the ability to accurately observe, analyze, and respond to a variety of different kinds of data for the purpose of continuously improving teaching and learning in the classroom and school. Data-literate teachers need not be experts in statistics or data collection methods. However, they do need to demonstrate three critical competencies: (1) the ability to make use of multiple data sources, (2) the skill to interpret data accurately, and (3) the capacity to engage in productive collaborative inquiry with their colleagues. • Formative assessments—assessments for learning that occur while lessons and learning are still underway—to diagnose student learn- ing needs, plan next steps in instruction, and provide students with descriptive feedback on how to improve their performance • Demographic data to identify characteristics of students, teachers, and the community • Data about people, practices, and perceptions to verify causes of student-learning problems and take effective action • Summative assessments—assessments of learning that happen after learning is supposed to have occurred—to determine whether learn- ing has taken place and to inform program changes • Achievement data disaggregated by race/ethnicity, gender, and eco- nomic, language, mobility, and educational status to uncover and address achievement gaps Interpreting Data Accurately • Distinguishing between observation and inference • Critically examining the assumptions and cultural biases that influence one’s data interpretations • Applying basic metrics accurately (e.g., percentage, percentile, percent- age change, and percentage point change) • Accurately interpreting line graphs, bar graphs, and scatter plots • Acting as critical consumers of tests based on an understanding of the importance of reliability (consistency, or likelihood of producing similar results again), validity (measuring what is intended), cultural sensitivity, and fairness Engaging in Collaborative Inquiry • Exhibiting habits of mind associated with productive collaborative inquiry (e.g., willingness to share practice with colleagues, continually learn, rely on data to test hypotheses, and take a moral stand for each and every student’s achievement) • Utilizing data as a catalyst to reflect on one’s own practice, not to blame students or their circumstances • Generating and testing out solutions to student-learning problems through frequently monitoring both the implementation (process) and the results (product) *Source: Adapted from: Love, N. et al.,(2008). The data coach’s guide to improving learning for all students: Unleashing the power of collaborative inquiry. Thousand Oaks, CA: Corwin, p. 129. The Data Pyramid* The Data Pyramid is a framework for answering these questions: (1) What kinds of data do teachers use? (2) For what purposes are the data helpful? and (3) How often should the data be analyzed? The front of the pyramid illustrates five different types of data that are important to school improvement. Each layer represents a different source of school data, with the width of the layer representing suggested relative frequency of use. The bottom and widest layer illustrates the data that teachers use most frequently, while the top layer shows the data source used least frequently. The side of the pyramid labeled “Frequency” offers rough guidelines, not rules, for how often teachers engage with each different type of data. TYPES OF DATA FREQUENCY Examples: District and state tests (disaggregate-, aggregate-, strand-, and item-level and student work) Demographic, enrollment, survey, interview, and observation data, curriculum maps End-of-unit, common grade-level tests reported at item level Math problems-of-the-week, writing samples, science journals, other student work Student self-assessments, descriptive feedback, use of rubrics/criteria, student products/performances, checking for understanding SUMMATIVE ASSESSMENT DATA ABOUT PEOPLE, PRACTICES AND PERCEPTIONS BENCHMARK COMMON ASSESSMENTS FORMATIVE COMMON ASSESSMENTS FORMATIVE CLASSROOM ASSESSMENTS 1 Time per Year 2-4 Times per Year Quarterly or End-of-Unit 1-4 Times per Month Daily-Weekly SAMPLE

DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

1

What is Data Literacy?

© 2011

Using Multiple Sources of Data

DATA LITERACY for Teachers

by Nancy Love

There is little dispute that educators’ effective use of school data is a hallmark of improving schools. But all the data in the world will have little impact on student achievement unless teachers feel comfortable, knowledgeable, and skilled in using a variety of data on a regular basis to improve teach-ing and learning. This reference guide provides a simple framework for strengthening data literacy and answering these questions: •Howcanschoolsmoveawayfromover-relianceonstateassessmentsandfromusingdataaseitheracarrotorastick? •Whatkindsofdatadoteachersanalyze,individuallyandwithcolleagues?Forwhatpurposes?Howoften? •Howcandatausehaveanimmediateanddirectimpactoninstructionandachievement?

Dataliteracyistheabilitytoaccuratelyobserve,analyze,andrespondtoavariety of different kinds of data for the purpose of continuously improving teaching and learning in the classroom and school. Data-literate teachers neednotbeexpertsinstatisticsordatacollectionmethods.However,theydoneed to demonstrate three critical competencies: (1) the ability to make use of multiple data sources, (2) the skill to interpret data accurately, and (3) the capacity to engage in productive collaborative inquiry with their colleagues.

•Formativeassessments—assessmentsforlearningthatoccurwhile lessonsandlearningarestillunderway—todiagnosestudentlearn- ing needs, plan next steps in instruction, and provide students with descriptive feedback on how to improve their performance•Demographicdatatoidentifycharacteristicsofstudents,teachers, and the community •Dataaboutpeople,practices,andperceptionstoverifycausesof student-learning problems and take effective action •Summativeassessments—assessmentsoflearningthathappenafter learningissupposedtohaveoccurred—todeterminewhetherlearn- ing has taken place and to inform program changes•Achievementdatadisaggregatedbyrace/ethnicity,gender,andeco- nomic, language, mobility, and educational status to uncover and address achievement gaps

Interpreting Data Accurately•Distinguishingbetweenobservationandinference•Criticallyexaminingtheassumptionsandculturalbiasesthatinfluence one’s data interpretations•Applyingbasicmetricsaccurately(e.g.,percentage,percentile,percent- age change, and percentage point change) •Accuratelyinterpretinglinegraphs,bargraphs,andscatterplots•Actingascriticalconsumersoftestsbasedonanunderstandingofthe importance of reliability (consistency, or likelihood of producing similar results again), validity (measuring what is intended), cultural sensitivity, and fairness

Engaging in Collaborative Inquiry

•Exhibitinghabitsofmindassociatedwithproductivecollaborative inquiry (e.g., willingness to share practice with colleagues, continually learn, rely on data to test hypotheses, and take a moral stand for each and every student’s achievement)•Utilizingdataasacatalysttoreflectonone’sownpractice,nottoblame students or their circumstances•Generatingandtestingoutsolutionstostudent-learningproblems through frequently monitoring both the implementation (process) and the results (product)

*Source:Adaptedfrom:Love,N.etal.,(2008).The data coach’s guide to improving learning for all students: Unleashing the power of collaborative inquiry.ThousandOaks,CA:Corwin,p.129.

The Data Pyramid*TheDataPyramidisaframeworkforansweringthesequestions:(1)Whatkindsofdatadoteachersuse?(2)Forwhatpurposesarethedatahelpful?and(3)Howoftenshouldthedatabeanalyzed?Thefrontofthepyramidillustratesfivedifferenttypesofdatathatareimportanttoschoolimprovement.Eachlayerrepresentsadifferentsource of school data, with the width of the layer representing suggested relative frequency of use. The bottom and widest layer illustrates the data that teachers use most frequently,whilethetoplayershowsthedatasourceusedleastfrequently.Thesideofthepyramidlabeled“Frequency” offers rough guidelines, not rules, for how often teachers engage with each different type of data.

TYPES OF DATAFREQUENCY

Examples:District and state tests (disaggregate-, aggregate-, strand-, and item-level and student work)

Demographic, enrollment, survey, interview, and observation data, curriculum maps

End-of-unit, common grade-level tests reported at item level

Math problems-of-the-week, writing samples, science journals, other student workStudent self-assessments, descriptive feedback, use of rubrics/criteria, student products/performances, checking for understanding

SUMMATIVE ASSESSMENT DATA ABOUT

PEOPLE, PRACTICES AND PERCEPTIONS

BENCHMARK COMMON ASSESSMENTS

FORMATIVE COMMON ASSESSMENTS

FORMATIVE CLASSROOM ASSESSMENTS

1 Time

per

Year

2-4

Times

per Year

Quarterly

or

End-of-Unit

1-4 Times

per Month

Daily-Weekly

SAMPLE

Page 2: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

2 © 2011

Formative Classroom Assessments: Using Data Day to DayThebaseofthepyramidillustratesthetypeofdatateachersshouldspendthebulkoftheirtimeusing— formativeclassroomassessments,orassessmentsforlearning.Examplesincludestudentself-assessments,descriptive feedback to students, use of rubrics with students, multiple methods of checking for understand-ing,quizzes,andexaminationofstudentwork.Formativeclassroomassessmentsinformteachers’instruc-tionaldecisions—daytoday,evenminutebyminute—andserveasthebasisforfeedbacktostudentstohelpthemimprovetheirlearning.Tomaximizethepowerofformativeassessmentstopromotelearning,students

need to be crystal clear on what the learning target is, what quality work looks like, where their work is now and, if there is a gap, how they can close it. Over 250 research studies from several countries establish that use of formative classroom assessments in these ways raises student achievement dramatically.Thatiswhyitishighlyrecommendedthatteachersspendmoretimeplanningfor,collecting,andanalyzingthesedatathananyother.Teachers use formative classroom assessments before, during, and after instruction, as illustrated here. Because teaching is really a three-part activity, involvingplanning,teaching,andreflecting,itiscriticalthatteachershavetimeonaregularbasisduringtheschooldaytodoallthree,bothindividuallyand with colleagues.

Formative Classroom Assessment Cycle: Using Data Day to Day

AdaptedfromResearchforBetterTeaching.(2010). Studying skillful teaching: Using data day to day(coursehandouts).Acton,MA:Researchfor Better Teaching, p. 143.

TheQuickSortisatoolthatteacherscanuseindividuallytoquicklyassessstudentneedsandplannextsteps.

AdaptedfromResearchforBetterTeaching.(2010).Studying skillful teaching: Using data day to day(coursehandouts).Acton,MA:ResearchforBetterTeaching,pp.149-152.

Mastery Objective:

Exceeds Meets Not Yet

Number of students in each category

Notes on errors, misconceptions, gaps, insights

BEFORE INSTRUCTIONTo Find Out What Students Know

•Pre-assess•Anticipateconfusionsandmisconceptions

AFTER INSTRUCTIONTo Reflect and Plan Next Steps

•UseQuickSorttodeterminewhosework exceeds, meets, or does not yet demonstrate masteryofobjective(seeQuickSortToolbelow)•Analyzestudentworkaccordingtorubricsor criteriaforsuccess(seeCriteriaAnalysisToolon page 3) •Identifymisconceptions,errors,andconfusion that need to be addressed•Plantoregroupstudentsandreteachorextend the next lesson based on evidence of what students need

DURING INSTRUCTIONTo Monitor and Adjust Teaching and Learning•Checkforunderstanding -Self-evaluation(e.g.,thumbsup;red,yellow, green cups or cards) - Direct content checks (e.g., handheld polling devices, individual whiteboards) •Makestudentthinkingvisible:Askstudentsto express their thinking verbally or in writing or by interacting with peers •Givestudentsspecific,descriptive(nonevalua- tive) feedback on how to improve their perfor- mance•Havestudentsself-assessandsetgoals•Regroupstudentsandreteachonthespotas needed

Formative Classroom

Assessment Cycle

Quick Sort Tool Step1:Giveaquickquizorbriefwrittenassignmenttodeterminetowhatextentstudentshavemetthelesson’smasteryobjective Step2:Sortworkintothreepiles—exceeds,meets,andnotyet Step3:Recordthenumberofstudentsineachcategory,notingcommonerrorsandyourowninsights,usingthetablebelow Step4:Reflectonfindingsandplanwhatyouwilldoforstudentsineachofthethreegroupsandtoaddresserrorsorconfusion

FORMATIVE CLASSROOM

ASSESSMENTS

SAMPLE

Page 3: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

3 © 2011

Formative Common AssessmentsThe second layer of the pyramid represents formative common assessments. These include some of the samesourcesofdataastheformativeclassroomassessments—quickquizzes,studentwritingsamples,socialstudiesprojectsinprocess,sciencejournals—thedifferencebeingthatteamsofteachersadministertheseassessmentstogetherandanalyzethemindatateamswiththeircolleagues.Forexample,alltheninth-grade algebra teachers administer the same open-response problem-of-the-week, using a common ru-

bricforscoring.Attheirweeklymeeting,theyanalyzestudentworkforpatternsofstrengthsandweaknessesandbrainstormwaystoadjusttheirinstructionandtoprovidedescriptivefeedbacktostudents.Attheirnextteammeeting,theywillseehoweffectivetheir strategies were by examining the results of the next problem-of-the-week, sharing best practices, and continuing the cycle of improvement. AnothermethodforanalyzingformativecommonassessmentsistheCriteriaAnalysisTool,typicallyusedbyteamsofteachersteachingthesamegradelevel(elementary)orsubjectorcourse(middleorhighschool).ThestepsfortheCriteriaAnalysisareasseeninthisexamplethatfollows:

Criteria Analysis ToolLesson Objective:Studentswillbeabletowriteafive-paragraphpersuasiveessayusingdetailsandrefutingcounter-argumentsinsupportofaclearly stated position. Product/Performance:Writefive-paragraphpersuasiveessayononeofthetopicsprovided. +=Exceedscriterion P=Proficientperformanceofcriterion -=Notyetproficient

Agreetouseacommonformativeassessmenttodeterminethe extent to which students meet the learning objective of a lesson or group of lessons. Determine the criteria for success for the assessment, i.e., descriptorsofproficientperformancewritteninstudent-friendlylanguage, such as:Elementary: Your journal entry includes: A.Theformatofaletter B.Adescriptionofwhatyousaw,heard,andtastedatthe marketplace C.Anillustrationthatmatchestheentry D.CorrectspellingofvocabularywordsMiddleSchool:(Seesampleabove)Yourfive-paragraph persuasive essay includes: A.Apositionstatement B.Atleastthreesupportingdetails C.Onetotworefutationsofcounter-arguments D.Correctspelling,punctuation,andgrammarHighSchool:Yourgraphicorganizerincludes: A.Thecorrectgraphicformforan“effects”textstructure B.AtleastfourmajoreffectsoftheBerlinConference C.Atleastonepieceofevidenceforeachmainidea,in note form (detail) D.Atleastoneexplanationforeachpieceofevidence, in note form (detail)

Teach lesson, give assessment, and collect student work. Bring samples of student work to your data team meeting and togetheranalyzeeachpieceofstudentworkastowhetheritexceeds, meets, or does not yet meet each of the criteria for suc-cess.RecordfindingsontheCriteriaAnalysisTool.(Alternatively,individual teachers can do this in advance of the meeting and bring in selected, provocative pieces of student work for team analysis.) Analyzerootcausesof“not-yet”performancebyaskingques-tions such as: •Didstudentshavetheprerequisiteknowledge/skills? •Wasthereamisconceptionorerrorthatneedstobe unraveled? •Werestudentsconfusedaboutwhatthecriteriameant? •Wasinstructionmisalignedwiththelearningobjectivesor criteria? •Wouldadifferentinstructionalstrategyorexplanatory devicehavebeenabettermatchforsomestudents?Determine next steps for students who exceeded, met, and did notyetmeetthecriteria.Agreeasateamtotryoutanewstrategytogether, such as sharing examples of journal entries with the class and having students assess to what extent they meet the criteria.

Step 1

Step 2

Step 3Step 4

Step 5

Step 6

Source:AdaptedfromResearchforBetterTeaching.(2010).Studying skillful teaching: Using data day to day(coursehandouts).Acton,MA:ResearchforBetterTeaching,pp.161-164.

FORMATIVE COMMON ASSESSMENTS

Students’ NamesAdiCole

DanteEitanSimone

Total+P-

Criteria CRefutationof1-2

counter-arguments

+PP-P

# %

Criteria DCorrectspelling,

punctuation, grammar

+--P-

# %

Notes on Errors, Insights

Criteria APosition statement

PPPPP

# %

Criteria BAtleast3

supporting arguments

+--P-

# %SAMPLE

Page 4: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

4 © 2011

Benchmark Common AssessmentsThe next layer of the pyramid represents benchmark common assessments, given quarterly or at the end of a unit to assess to what extent students have mastered the concepts and skills in the part of the curriculum recently taught. These are administered together by teachers teaching the same content, eitheratthesamegradelevelorinthesamesubjectorcourse.The“common”featuremakesthemanidealsourceofdataforcollaborativeinquiry.Infact,theyareamongthemostimportantsourcesof student-learning data that a data team has because they are timely, closely aligned with local curriculum, and available to teachers at the item level (results are reported on each individual item, as in the sample report shown below) alongwiththeassessmentitemsthemselves(unlikesomestandardizedtests,wherereleasedtestitemsarenotavailable).Itemanalysisprovidesextremelyusefulinformationonstudents’misconceptionsandonthespecificconceptsorskillsthatstudentsneedhelpwith.

Asimplecolor-codingsystem,usingredtohighlightmost frequently missed items, yellow for caution areas, and green for items that meet expectations, canhelpteachersidentifystrengthsandprioritizestandards or types of items on which to focus. Teachers can set their own criteria or use district orstateguidelinesforthis“stoplighthighlighting.”Itis also useful to highlight in pink frequently chosen distractors or incorrect responses (i.e., 20% or more), which can point to student errors or miscon-ceptions to investigate further.Whilebenchmarkcommonassessmentsaresum-mative in that they provide evidence of learning after teaching, they also serve several formative purposes.Asteacherssharetheirresultsacrossgrade levels, they also share teaching strategies and materials that worked, providing immediate fodder for instructional improvement. Benchmark common assessments can also inform curriculum changes, such as increasing the amount of time a particular concept is taught or changing the sequenceinwhichitistaught.And,finally,theseassessments identify individual students who need extra help so that timely and targeted assistance can be provided.

Data About People, Practices, and Perceptions People: The next layer in the pyramid, data about people, practices, and perceptions, is one that is often overlooked in schools, but it is extremely important. Data about people include demographic data aboutstudentsandthecommunity,whichhelpeducatorsunderstandtheracial/ethnic,socio-economic,language, and special education populations that make up the school and the larger community, and changes in that community over time. Other valuable data about people include course-enrollment,

attendance,discipline,anddropoutrates.Whendisaggregatedbydemographicgroups,thesedatacanoftensurfaceinequitiesthatareimportanttouncoverandconfront,suchasunder-representationofminoritystudentsinhonorsandAdvancedPlacementclasses.

Practices:Data-savvyeducatorsrecognizetheimportanceofex-amining data, not just about students, but also about the adults in the schoolandtheirpractices.Forexample,teachersonafourth-gradedata team agree to ask more open-ended questions in classes as a strategy for increasing rigor, student engagement, and achievement. Then they keep track each week of how many such questions they asked.Whentheirdatateammeets,theycollectandanalyzethesedata about their own instructional practice to track progress. Or, as illustrated in the graph on the top of page 5, a third-grade data team sets a goal of increasing the amount of time they spend engaging students with non-routine, open-response mathematics problems and

trackstheirmonthlyprogress.Dataaboutpractice—whetheritishowfrequentlyteachersarecheckingforunderstandingorteachingnonfic-tion writing across the curriculum or to what extent they are implement-inganewinquiry-basedscienceprogram—helptoidentifyunderlyingcausesofstudent-learningproblems.Inaddition,thesedataprovideaway to monitor instructional improvement as an interim and necessary stepontheroadtoimprovingachievement.WhiletheDataPyramidsuggests that data about people, practices, and perceptions may be used two to four times a year, data about practice may be used more frequentlywhenmonitoringaspecificinstructionalpractice.

continued on next page...

BENCHMARK COMMON

ASSESSMENTS

DATA ABOUT PEOPLE, PRACTICES, AND PERCEPTIONS

Sample Excerpt: Benchmark Common Assessment ReportHigh School Biology: 3rd Quarter

Bolded percentage = correct response Number of students = 130 Criteria For Stoplight Highlighting Multiple-Choice Items

80-100% correct 70-79% correct Below 70% correct 20%+ chose the same incorrect response

*Item Type:MC = Multiple Choice (1 pt ea.); OR = Open Response (4 pts ea.); SA = Short Answer (1 pt ea.)

% of School’s Total Student Responses (for each MC answer or score point for SA or OR)Item Type* Standard Blank/0 A/1 B/2 C/3 D/4

Pink:Green: Yellow: Red:

00000000

5255728

100

667

14832859

48451424

3833152277

24

00

3414

7143148431714

003414

79302807483

001021

1 MC B.12 MC B.23 MC B.44 MC B.35 MC B.56 MC B.67 MC B.48 MC B.3

21 SA B.122 SA B.523 OR B.324 OR B.4SAMPLE

Page 5: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

5 © 2011

Summative AssessmentThe top layer of the pyramid represents summative assessment data, including state assessments and annual district tests. These data are used for accountability purposes and to determine if student outcomeshavebeenmet.However,theyalsocanprovidevaluableinformationabouthowtoimprovecurriculumandinstructionandbetterserveunderservedstudents.Especiallywhenusedinconjunc-tion with other data sources previously described, summative assessments guide the development of annual school and district improvement targets and plans.

Perceptions: Dataaboutperceptions—whatkeystakeholdersinaschool(parents, administrators, teachers, paraprofessionals, and students) think and feel—areanothervitalsourceofinformationforteachers.Collectedthroughsurveysand interviews, perceptual data sheds light on topics such as student engage-ment,senseofbelonging,oropportunitiestolearn.Fromparents,teacherscanlearn how welcome parents feel in the school or how well they understand the new gradingsystem.Fromteachers,administratorscanlearnhowpreparedteachersfeel to teach the new inquiry-based science program and what kind of professional development and support they need. Perceptual data can be very important when teachersareexploringcausesofstudent-learningproblemsidentifiedinachieve-mentdata.Theyalsohelptoassurethatdiversevoices—byrole,byrace/ethnicity,andbyeconomic,language,andeducationalstatus—informeducators’decision-making.Whileparentorstudentsurveysmightbeanalyzedonceortwiceayear,teachers can check in regularly with their students about how they are experiencing the classroom climate and instruction.

To take full advantage of summative assessments, it is important to drilldownintothem,analyzingtheminasmuchdetailaspossible,preferably in a data team or faculty meeting setting:•Attheaggregatedlevel,lookforbigtrends,suchaschangesover timeinthepercentageofstudentsachievingproficiency.•Atthedisaggregatedlevel(brokenoutbystudentpopulations, e.g.,race/ethnicity,gender,andeconomic,language,mobility, and educational status), examine trends over time related to achievement gaps.•Atthestrandlevel(clustersofrelatedcontentandskills,suchas algebra, geometry, number sense, and statistics and probability in mathematics), look for strengths and weaknesses in student achievement in particular content strands.•Whendataarereporteditembyitemand/orbyindividualstan- dards, pinpoint standards and types of items, such as extended- response, with which students struggle.•Ifactualstudentworkismadeavailablebythetestingcompany, such as student responses to writing prompts or other open- response items, these provide a rich source of evidence of students’ thinking, common misconceptions, and gaps in skills and under- standings that can be addressed in next year’s instruction.

Summative assessments occupy a small part of the pyramid because they are only available an-nually and provide limited information about what to do to improve (especially if item-level data are not available). Inaddition,becauseof the timing of the results, these data are often more of an autopsy report than a diagnosis. The inverted pyramid illustrates the practice that we recommendschoolsmoveawayfrom—spendingmostoftheirtimeand energy on state assessments rather than on formative assess-ments and data about people, practices, and perceptions.

Data About Practices: Implementation Monitoring

AdaptedfromResearchforBetterTeaching.(2010).Unleashing the power of collaborative inquiry: A program for data coaches (course handouts).Acton,MA:ResearchforBetterTeaching,p.383.

AverageTime Spenton Mathematics Problem- Solving PerWeek

SimSchool– Grade3

Goal:45 Minutes/Week

Responding to Data — Looking for Love in All the Right PlacesKnowing what kinds of data to use for what purposes is an important dimension of skillful datause.Equallyimportantisreflectingonandquestioning those data and responding in ways that have a direct and immediate impact on studentsandtheirlearning.Forexample,whendata reveal content standards that many students are not mastering, educators should “look for love inalltherightplaces,”thatis,tolookforcausesof student-learning problems in those areas of practice known to impact student achievement:

Didweteachit?Inenoughdepth?Placedintherightsequence?Frequentlyenough?Didweuseavarietyofresearch-basedinstructionalapproaches?Arewesharingsuccessfulpractices?Didwereteachusingadifferentapproachtoindividualsorgroupswhodidn’tyetgetit?Did we use ongoing formative assessment to explore student thinking and build on it in our instruction?Communicatetostudentshowtoimprove?Helpthemself-assess?Didweexamineattitudes/practicesthatmightcontributetoinequitiesinachievement, relationships,orqualityofinstruction?Didallstudentshaveequalaccesstorigorouscontent/high-expectationsteaching?Didweidentifystudentswhoneedadditionalhelpandprovidethemwithit?

Whatknowledgeorskillswouldhelpustobetterteachthiscontent?

CurriculumInstruction

Assessment

Equity

Individual AssistanceProfessional Development

Inverted Data Pyramid: Not Recommended

FromN.Love,K.Stiles,S.Mundry,andK.DiRanna.(2008).The data coach’s guide to improving learning for all students: Unleashing the power of collabora-tive inquiry.ThousandOaks,CA:CorwinPress, p.132.Allrightsreserved.

Summative District or State Assessments

SUMMATIVE ASSESSMENT

SAMPLE

Page 6: DATA LITERACY - NPRInc Blogblog.nprinc.com/downloads/data-literacy-for-teachers-dlft.pdf · Data literacy is the ability to accurately observe, analyze, and respond to a variety of

Price:$12.95Author: Nancy Love © 2011 Layout&Design:AndreaCeroneAllrightsreserved.Nopartofthispublicationmaybe reproduced or transmitted in any form, or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without written permission from the publisher. Dude Publishing, an imprint of NationalProfessionalResources,Inc.

6

National Professional Resources, Inc.25 South Regent StreetPort Chester, NY 10573

1-800-453-7461 ~ www.NPRinc.com

Order From:

Often responding to data requires that teachers teach a skill or concept again for students who have not yet mastered it. But teaching it again the same way is not likely to produce a different result. The acronym below suggests multiple ways teachers can reteach or extend lessons:

Teach concept or skill in a different way, e.g., modeling thinking aloud, mental imagery, a physical demonstration, or a computer simulation

Engagelearnersinadifferentway,e.g.,usingcooperativegroups,learningcenters,movementactivities,activators,summarizers,graphicorganizers

Alignreteachingtotheessenceoftheerrororconfusion

Challengestudentswithamorerigorousproblemortask

Helpstudentswhoneeditwithassistanceoutsidetheclassroomandmoreopportunitiesforpractice

TEACH

Professional Development Programs Studying Skillful Teaching: Using Data Day to Day presents an overview oftheknowledgebaseoneffectivepedagogy,emphasizinguseofavarietyofdata—pre-assessments,checkingforunderstanding,ongoingformativeassessments,andstudentwork—toinformplanning,teaching,andreflectingonlessons.37.5-hourprogram.Unleashing the Power of Collaborative Inquiry: A Program for Data Coaches builds the capacity of teachers and coaches to lead collabora-tive inquiry with school-based data teams and to use data continuously, collaboratively, and effectively to improve teaching and learning. 37.5-hour program with onsite coaching.

Contact: ResearchforBetterTeaching OneActonPlace,Acton,MA01720 978-263-9449•www.RBTeach.com•[email protected]

References and Resources Black,P.,Harrison,C.,Lee,C.,Marshall,B.,&Wiliam,D.(2004,Sep- tember).Workinginsidetheblackbox:Assessmentforlearningin the classroom. Phi Delta Kappan,86(1),9-21.Guskey,T.(2007,December–2008,January).Therestofthestory. Educational Leadership, 65(4),28-35.Love,N.(ed.).(2009).Using data to improve learning for all: A collabora- tive inquiry approach.ThousandOaks,CA:Corwin.Availableat www.RBTeach.com.Love,N.,Haley-Speca,M.,&Reed,D.(2010,Spring).Theskillfuldata team:Fouressentialsforimprovingteachingandlearning.Per- spectives(MassachusettsAssociationforSupervisionandCurricu- lum Development), 2-5. Love,N.,Stiles,K.E.,Mundry,S.,&DiRanna,K.(2008).The data coach’s guide to improving learning for all students: Unleashing the power of collaborative inquiry.ThousandOaks,CA:Corwin. Availableatwww.RBTeach.com.Popham,W.J.(2008). Transformative assessment.Alexandria,VA: AssociationforSupervisionandCurriculumDevelopment.ResearchforBetterTeaching.(2010).Unleashing the power of collab- orative inquiry: A program for data coaches (course handouts). Acton,MA:ResearchforBetterTeaching.ResearchforBetterTeaching.(2010).Studying skillful teaching: Using data day to day(coursehandouts).Acton,MA:ResearchforBetter Teaching. Saphier,J.,Haley-Speca,M.A.,&Gower,R.(2008).The skillful teacher: Building your teaching skills, 6th ed. Acton,MA:ResearchforBetter Teaching.Availableatwww.RBTeach.com/Stiggins,R.,Arter,J.,Chappius,J.,&Chappius,S.(2006).Classroom assessment for student learning: Doing it right—Using it well. Princeton,NJ:EducationalTestingService.Stiggins,R.,&DuFour,R.(2009,May).Maximizingthepowerofforma- tive assessments. Phi Delta Kappan,90(9),640-644.

Whenteachersfeelconfidentandcapableofusingdatainthewaysdescribed above, they unleash the power of data to improve teaching andlearning.Schoolshavemoreandmoredataandincreasingpres-surestoimprove.Butneitherissufficient.Whatisneedediswidespreadcapacity—sothateveryteacherknowshowtousedatacontinuously,collaboratively, and effectively. The problems in schools are too compli-cated and complex to be solved by a few leaders or by use of only one datasource—stateassessments—onceayear.Itisteachers,usingdataday to day, individually and with their colleagues, who turn the wheels ofschoolimprovement.Widespreaddataliteracyispreventivemedicineagainst ineffective uses of data that have dogged schools over the last decade.

Why Widespread Data Literacy Is Important

Acknowledgment: The author gratefully acknowledges the con-tributions of her colleagues at Research for Better Teaching, her co-authors of the Data Coach’s Guide (K.E. Stiles, S. Mundry, and K. DiRanna), and the educators with whom I have partnered to develop the ideas contained in this guide.

Reteach Using T-E-A-C-H

SAMPLE