chapter 15 Systems for State Science Assessment: …...Assessment: Findings of the National research council’s committee on Test Design for k–12 Science Achievement Meryl W. Bertenthal

c h a p t e r 1 5

301

Systems for State Science Assessment: Findings of the National research council’s committee on Test Design for k–12 Science AchievementMerylW.BertenthalIndiana University

MarkR.WilsonUniversity of California, Berkeley

AlexandraBeattyNational Research Council

ThomasE.KellerNational Academy of Sciences1

Shortly after the passage of the No Child Left Behind Act of 2000(NCLB), theNationalScienceFoundationaskedtheNationalRe-searchCouncil (NRC) to formacommittee toprovideadviceand

guidance to states and their testdevelopmentcontractorson thedesignand implementation of quality science assessments to meet the require-ments of the law. This chapter summarizes some of the findings and

1Meryl Bertenthal served as the study director and Mark Wilson was the chair of the com-mittee that authored the report on which this chapter is based. Alexandra Beatty served as a program officer to the committee. Thomas Keller served as the chair of the state science supervisors’ panel that advised the committee.

Copyright © 2008 NSTA. All rights reserved. For more information, go to www.nsta.org/permissions.

302 N A T I o N A l S c I e N c e T e A c H e r S A S S o c I A T I o N

cHAPTer15 SecTIoN 3: HIgH-STAkeS ASSeSSmeNT

recommendationsoftheNRCCommitteeonTestDesignforK–12ScienceAchievementand isbasedonthecommittee’s final,book-lengthreport,Systems for State Science Assessment(NRC2006).2Whilethecommittee’sreportistargetedatstatepolicyandeducationdecision-makers,manyofthefindingsareofinteresttoteachersandothersinvolvedintheimprove-mentofscienceeducationandassessmentintheirstates.Inthischapterwetrytoconnectafewofthecommittee’sfindingsandrecommendationstoissuesofinteresttoteachers. Thereareinherentdifficultiesinsummarizingawholebookinasinglechapter.Onesuchdifficultyisprovidingthekindofspecificdetailandexem-plificationthatthereadermightbelookingfor.Readerswhoseekadditionalexamplesorwhoareinterestedinlearningmoreaboutthesuggestionsandapproachesmentionedhereareencouragedtoconsultthecommittee’sfullreport,whichisavailablefromtheNationalAcademiesPress.3 ManyofthepointsmadeintheNRCreportandinthischaptermayap-plytoeducationalassessmentinotherareas.Inparticular,theprinciplesthatframedthecommittee’s thinkingaboutscienceassessmentcouldguideas-sessmentinothersubjectareasaswell.However,thereareaspectsofscienceasadiscipline—forexample, theabstractnatureofmanyof theconceptsthatstudentsareexpectedto learnandtheemphasisonscientific inquiryandinvestigationinmanystatestandards—thatpresentchallengesspecificto scienceassessment.Thus, indesigninghigh-quality scienceassessment,stateswillneedtofocusonboththegeneralpreceptsofsoundeducationalmeasurementandthefeaturesthatareuniquetoscienceassessment.

no child Left Behind and Science assessmentNCLBisanextensionofthe1994reauthorizationoftheElementaryandSecondaryEducationAct(ESEA).(ESEAmovesbeyondthatlawbecauseitaffectsallpublicschools,districts,andstudentsandbecauseitincludesscienceinitsrequirementsforstandardsandassessments.) NCLBrestsonfourpillars:(1)asetofchallengingcontentandachieve-ment standards for studentperformance; (2) assessments tomeasure the_______2Wearegratefultothecommitteemembersandtotheadvisorygroupsofscienceteachers,

scienceeducationsupervisors,andstateassessmentdirectorsthatassistedthecommit-teefortheircontributionstothereport.Weacknowledgeandthankthemfortheirintellectualcontributiontothischapter.

3Thereportcanbeviewedonlineathttp://books.nap.edu/catalog/11312.html


303303A S S e S S I N g S c I e N c e l e A r N I N g

cHAPTer15SecTIoN 3: HIgH-STAkeS ASSeSSmeNT

achievementofallstudentsrelativetothestatestandards;(3)flexibilityforstatesanddistrictstoimplementinstructionandcurriculumsothatallstudentscanachievethestandards;and(4)accountabilitymeasuresthatmoveschoolstowardgreatereffectiveness inpromotingachievement. Inthischapter,asinthecommittee’sreport,thefocusisonthefirsttwopil-lars: standards and assessment. UnderNCLB,statesarerequiredtoraisethereading,mathematics,andscienceachievementlevelofallstudentsandtonarrowtheachievementgap inthesesubjectsamongstudentsofdifferentbackgrounds.Toattainthesegoals,statesarerequiredtohaveasetofchallengingstandards,ex-pand their standardized testing programs, analyze and report assessmentresultsinspecificwaysforaccountabilitypurposes,andensurethatallstu-dentshavetheopportunitybothtobetaughtbyqualifiedteachersandtolearninwell-performingschools. NCLBputsinplacestrongaccountabilitymeasuresforschoolsanddis-trictsandimposessanctionsonschoolswhentheirstudentsdonotmakeadequateyearlyprogress(AYP).Sothatstatesworkdiligentlytowardthesegoals,thereisarigoroustimetableforstatecompliancewithNCLBregula-tions.Currently,scienceisnotpartofAYPformulas,butNCLBrequiresthatresultsfromscienceassessmentsbereportedpubliclyalongwithotherinformationonstudentachievement.4 This means that science assessment under NCLB will still carry high stakes for schools and school districts.Individualsandschoolswillgettheirownresults,andthey,aswellasoth-erstakeholders,willbeabletocomparetheir resultswiththoseofotherschools. It was the consensus of the NRC committee that teachers willtailortheirinstructiontotheassessmentsand,therefore,thequalityoftheassessmentswillbeofparamountimportance. Under NCLB, states must have challenging academic content andachievementstandardsforscienceinplaceby2005–2006.Statesmustbe-ginmeasuringstudentattainmentoftheirstandardsin2007–2008throughassessmentsthatarefullyalignedwiththestandards.Theassessmentsmustmeetacceptedprofessionalstandardsfortechnicalqualityforthepurposes_______4Asthischapterwasbeingcompleted,severalbillswerebeingintroducedinCongressto

requiretheinclusionofscienceassessmentresultsinAYPformulas.Thecommitteethatauthoredthereportonwhichthispaperisbasedsuggestedthatsuchastepshouldnotbeundertakenwithout careful considerationofboth thepositive andnegativeimplicationsofincludingorexcludingsciencefromAYPformulas.




forwhichtheywillbeused.Thecommitteefoundthatmeetingthetechni-calqualityandalignmentprovisionsofthelawcouldpresentseriouschal-lengesforstates.Forexample,thecommitteefoundthatsomestatesmaylist somanystandards that testscannotmeasure themall.Further,moststateassessments fail tomeasureadequately thecognitivecomplexityor“depthofknowledge”describedinstatestandards. NCLBalsospecifiesthatstates’assessmentsystemsmustincorporatemul-tipleup-to-datemeasuresofstudentachievement,includingmeasuresthatassesshigher-orderthinkingskillsandunderstandingofchallengingcontent.Scienceassessmentsaretobeadministeredannuallytoallstudents,includ-ingthosewithdisabilitiesandthosewhoarenotfluentinEnglish,atleastonceineachofthreegradebands,3–5,6–9,and10–12.StatesarerequiredtomakereasonableaccommodationsforstudentswithdisabilitiesandlimitedEnglishproficiencytoallowthemtoparticipateintheassessments,andtheymusthaveinplacealternateassessmentsforstudentswhocannotparticipateintheregularassessmentevenwithaccommodations.[FormoreinformationaboutissuesrelatedtoassessingstudentswithdisabilitiesorEnglishlanguagelearnerssee,forexample,Keeping Score for All (NRC2004); Testing English-Language Learners in U.S. Schools(NRC2000);andpages136–141ofthecommittee’sreport(NRC2006).] Under NCLB, states’ assessment systems may include many differenttypesofassessment strategiesandbecomprisedofauniformsetofassess-mentsstatewideoracombinationofstateandlocalassessments.Regardlessoftheformthatastate’sassessmentsystemtakes,theresultsmustbereportedpubliclyandbeexpressedintermsofitsacademicachievementstandards.Theresultsmustbereportedintheaggregateforthefullgroupoftesttakers,be disaggregated for specified population groups, and provide informationthatisdescriptive,interpretive,anddiagnosticattheindividuallevel.

StandardsContentstandardsareconsideredfundamentaltoscienceeducationandsci-enceassessmentbecausetheydefinewhatistobetaughtandwhatkindofperformanceisexpected.Forstandardstoservethisfunctionwell,theymustcommunicate clearly to everyone concerned—teachers, assessment devel-opers,students,parents,andpolicymakers—whatstudentsareexpectedtoknowandbeabletodo.TheNRCcommitteefoundthatstandardswillcom-municatebestiftheyareclear,detailed,andcomplete;reasonableinscope;




rigorousandscientificallycorrect;andbuiltaroundaconceptualframeworkthatreflectssoundmodelsofstudentlearning(NRC2006,Chapter4). Thecommitteealsofoundthatmanystatecontentstandardsand,con-sequently, thecurriculaaligned to themcontain toomanydisconnectedtopics.Theyfoundthatthereisnotenoughattentionpaidtohowastu-dent’sunderstandingofatopiccanbesupportedandenhancedfromgradeto gradeor across subjectdisciplines.As a result, topicsoften appear toreceive repeatedcoverageovermultipleyearswith fewopportunities forstudentstoseeconnectionsbetweenatopicunderstudyandotherrelatedtopics, thusgivingstudentsanincomplete foundationfor furtherknowl-edgedevelopment. Ingeneral,goodstatestandardsshouldbeorganizedandelaboratedinways that clearly specifywhat studentsneed toknowandbeable todoandhowtheirknowledgeandskillswilldevelopovertimewithcompetentinstruction. Science standards should be structured around central prin-ciplesofsciencethatrepresentthefoundationfortheconcepts,theories,principles,andexplanatoryschemasforphenomenainadiscipline.Thesecentralprinciples,whicharesometimesreferredtoasthe“bigideas”ortheenduringunderstandings(seeWigginsandMcTighe1998)ofscience,arethefoundationfortheprinciples,theories,andexplanatoryschemeswithinadiscipline.Whilethereisnouniversallyacceptedlistof“bigideas,”andstatesmayhavetodecideontheirindividualfoci,possibleexamplesmightbeevolution,Newton’slaws,orkineticmoleculartheory. Organizingstandardsaroundbigideasisafundamentalshiftfromtheor-ganizationalstructurethatmanystatesuseinwhichstandardsaregroupedundersubjectareasordiscretetopics.Thecommitteefoundthatapoten-tialpositiveoutcomeofthereorganizationofstatestandardsfromdiscretetopicstobigideasistheopportunityforashiftinemphasisfrombreadthofcoveragetodepthofcoveragearoundarelativelysmallsetoffounda-tionalprinciplesandconcepts.Thesefoundationalprinciplesandconceptsbecomethefocusedtargetofinstructionandcanbeprogressivelyrefined,elaborated,andextendedovertime(NRC2006,p.3). The committee identified twoideasthatemergefromthescienceedu-cationresearchliterature—learningprogressionsandlearningperformanc-es—thatwouldbeusefulforstatestoadaptinorganizing,elaborating,andcritiquingtheirstandardsinordertomakethembetterabletoguidecur-riculum,instruction,andassessment.Thesetwoideaswouldalsobeuseful




forteachers,curriculumdesigners,andassessmentdevelopersbecausetheyprovidethespecificityandorganizationalframeworkthatismissinginmoststandardsdocumentsbutthatisneededforcoherenceamongcurriculum,instruction,andassessment.

Learning ProgressionsLearningprogressions,whichcanbedeveloped for lessons,unitsof study,yearlongcourses,oranentireK–12experience,describeinwordsandexam-pleswhatitmeanstodevelopgreaterunderstandingofanideaorsetofideas.Theyareanchoredononeendbystudents’priorknowledgeandontheotherbytheexpectationsfor learningovertimewithinstruction.Learningpro-gressionsproposetheintermediateunderstandingsorbenchmarksbetweenanchorpointsthatcontributetobuildingamoresophisticatedunderstand-ingandserveastargetsforcurriculum,instruction,andassessment. Thecommitteeemphasizesthatlearningprogressionsarenotdevelop-mentallyinevitable;morethanonepathleadstocompetence.Thepath-waysthatindividualstudentsorgroupsofstudentsfollowdependonmanythings,includingtheknowledgeandexperiencethattheybringtothetask,thequalityandcontentoftheinstructionthatsupportstheirlearning,andthenatureofthespecifictasksthatarepartoftheexperience.Nonetheless,somepathsarefollowedmoreoftenthanothers.Organizingstandardstoreflectthemoretypicalwaysinwhichdeepeningunderstandingdevelopsprovidesastructurearoundwhichstates,curriculumdevelopers,andteach-erscanorganizelearning.Also,thisorganizationcanprovidecluesaboutthetypesofassessmenttasks thatcanbeusedto shed lightonstudents’achievementatdifferentpointsalongtheprogression. Ideally, learning progressions should be based on research about howcompetencedevelopsinaparticulardomain.However,formanyaspectsofsciencelearning,theresearchliteratureisincompleteandresearchfindingsmayhavetobesupplementedwiththejudgmentsofexpertteachersandotherswithknowledgeofhowstudentslearnscienceandwhatisknownmoregenerallyaboutcognitionandsciencelearning. Whatmightalearningprogressionlooklike?Figure15.1showsalearn-ingprogressionforatomicmoleculartheorythatwasdevelopedasanex-amplefortheNRCcommittee(Smithetal.2006).




The learning progression is organized around two sets of big ideas. The first set (M1, M2) consists of big ideas about (a) the properties of matter and material kinds, (b) their constancies and changes across transformations, and (c) the role of measurement, modeling, and argument in knowing. This first set of big ideas can be introduced in the earliest grades and elaborated throughout school-ing at the macroscopic level. For example, students move from describing the properties of material kinds to learning about density as the ratio of weight over volume, and from conservation of material kind and weight during melting to conservation of mass across all phase changes. In middle school and high school, the atomic-molecular theory can be introduced as a second set of big ideas. This second set of ideas (AM1, AM2, AM3, AM4) builds on the first because understanding the atomic-molecular theory depends to a great extent on the macroscopic big ideas studied earlier (e.g., the understanding that matter has weight and occupies space) and, at the same time, it provides deeper explana-

Figure 15.1 learning Progression for Atomic molecular Theory

Six Big Ideas of Atomic Molecular Theory That Form Two Major Clusters

M1. Macroscopic Properties:Wecanlearnabouttheobjectsandmaterialsthatconstitutetheworldthroughmeasurement,classification,anddescriptionaccordingtotheirproperties.M2. Macroscopic Conservation: Mattercanbetransformed,butnotcreatedordestroyed,throughphysicalandchemicalprocesses.AM1. Atomic-molecular theory: AllmatterthatweencounteronEarthismadeoflessthan100kindsofatoms,whicharecommonlybondedtogetherinmoleculesand networks.AM2. Atomic-molecular explanation of materials:Thepropertiesofmaterialsaredeterminedbythenature,arrangement,andmotionoftheatomsandmoleculesof which they are made.AM3. Atomic-molecular explanation of transformations:Changesinmatterinvolvebothchangesandunderlyingcontinuitiesinatomsandmolecules.AM4.Distinguishingdatafromatomic-molecularexplanations:Thepropertiesofandchangesinatomsandmoleculeshavetobedistinguishedfromthemacroscopicpropertiesandphenomenatheyaccountfor.




tory accounts of macroscopic properties and phenomena. They also enable the further elaboration of macroscopic understandings. (p.12)

Learning PerformancesThecommittee found thatmost state science standardsdescribe sciencecontent knowledge—and the general cognitive demands relative to that knowledge—withoutprovidingoperationaldefinitionsofwhatitmeanstoknoworunderstand.Therefore,standardsmustbeelaboratedbeforetheycanserveasabasisforinstructionorassessment.Iftheyarenotelaboratedclearly,teachersandassessmentdeveloperscaninferfromalmostanystan-dardanarrayofmeanings thatdrawonmanydifferentcombinationsofcontent knowledge and cognitive demand. Learning performances are a way to reformulate a scientific contentstandardintermsofthescientificskillsandpracticesthatuse that content, suchasbeingabletodefineterms,describephenomena,usemodelstoex-plainpatternsindata,constructscientificexplanations,ortesthypotheses(Perkins1998;Reiseretal.2003).Bydefiningthemost importantskillsand practices for which the knowledge is used, it is possible to connecttheconceptualstatementsinthecontentstandardswithassessmentperfor-mancesinwhichstudentscandemonstratetheirunderstanding. Articulatedlearningperformancesactasguidesfordesigningassessmentsandcanserveasthelinkbetweenclassroomandlarge-scaleassessmentsbydefiningwhatstudentswhomeetthestandardwouldbecapableofdoingwith theirknowledge.Assessment tasks could thenbedeveloped specifi-callytoelicitevidencethatstudentscanusetheirknowledgeasdescribedinthestandard.Learningperformancescanbetargetedbroadlybystateanddistrict tests and at a more fine-grained level in classroom assessments.

assessmentAssessmentisasystematicprocess forgatheringinformationaboutstu-dentachievement.Itprovidesimportantinformationformanydifferentpurposesthatare importanttotheeducationsystem, includingguidinginstructionaldecision-makingintheclassroom,holdingschoolsaccount-able for student achievement, and monitoring and evaluating educa-tionalprograms. It isalsothewaythatteachers, schooladministrators,andstatepolicy-makersexemplifytheirgoalsforstudentlearning.Whileassessment can do all of these things, it must be designed specifically




toservetheparticularpurposeorpurposesforwhichtheresultswillbeused.Forexample,anassessmentthatisdesignedtoprovideinformationaboutstudentachievementthatisdiagnostic,descriptive,andinterpre-tive would need to test students’ understanding of a few key conceptsdeeply and thoroughly. On the other hand, an assessment that is usedtoinformpolicy-makersabouttheeffectivenessoftheoveralleducationsystemwouldneedtocoverbroadlyallofthetopicsdeemedimportantbydecision-makers.Neitherofthesestrategiescouldprovideresultsthatarevalidforthepurposesoftheother(NRC2006,Chapter2).

A System of AssessmentThecommitteeconcludedthatasingleassessmentstrategywouldnot,byitself,meettherequirementsofNCLBasitcouldnotprovideresultsthatwouldbevalidandreliableforallofthepurposesidentifiedunderthelaw.Thecommitteethereforerecommendedthatstatesdevelopasystemofsci-enceassessmentthatcollectivelywouldmeetthevariouspurposesofNCLBandprovideeducationdecisionmakerswithassessment-basedinformationthatisappropriateforeachspecificpurposeforwhichitwillbeused.ThesystemthateachstatedevelopsinresponsetoNCLBnecessarilywillvaryaccordingtothestate’sgoalsandprioritiesforscienceeducationanditsusesforassessmentinformation.Forexample,astatemightchoosetodevelopasingletestinwhichstudentstakeacommoncoreassessmentthatpro-videsindividualresults,alongwithanassessmentwithamatrix-samplingdesignthatprovidesinformationabouttheachievementofgroupsofstu-dentsacrossabroadcontentdomain.Orastatemightchoosetocombinestandardized classroom assessments that provide diagnostic, descriptive,andinterpretiveinformationwithanexternalassessmentthatshowstheprogressthatallstudentsaremakingtowardachievingstatestandardsorforprogramevaluationpurposes.Orastatemaydecidetogiveupastatewidetestand insteaduseoneofmanydifferentmodels inwhich results fromlocal,district,orstateassessmentsarecombined,aggregated,andreportedforspecificpurposes.5Whenmultipleassessmentstrategiesareused,theyshouldbedesignedfromthebeginningto functionaspartofacoherentsystem of assessment. 5Examplesofthesearesummarizedinthecommittee’sreport(pp.31–37)andareillus-

tratedindetailinpaperscommissionedbythecommittee.Thesepapersareavailableonlineat:www7.nationalacademies.org/bota/Test_Design_K-12_Science.html




Acoherentsystemofstandards-basedscienceassessmentishorizontallycoherent:curriculum,instruction,andassessmentareallalignedwiththestandards; target the same goals for learning; and work together to sup-portastudent’sdevelopingscienceknowledge,understandings,andskills.It is vertically coherent: All levels of the education system—classroom,school,schooldistrict,andstate—arebasedonasharedvisionofthegoalsforscienceeducation,ofthepurposesandusesofassessment,andofwhatconstitutes competent performance. The system is also developmentallycoherent:Ittakesintoaccounthowstudents’scienceunderstandingdevel-opsovertimeandthescientificcontentknowledge,abilities,andunder-standingthatareneededforlearningtoprogressateachstageoftheprocess(Wilson2005). Ifacollectionoftestsisnotcoherent,theinformationthatisproducedcanyieldconflictingorincompleteinformationandsendconfusingmes-sagesaboutstudentachievementthataredifficulttountangle.Ifdiscrep-ancies in achievement are evident, it is hard to determine whether the testsinquestionaremeasuringdifferentaspectsofstudentachievement—andareusefulasdifferentindicatorsofstudentlearning—orwhetherthediscrepancyisanartifactofassessmentproceduresthatwerenotdesignedto work coherently together. Gaps in the information provided by theassessmentsystemcanleadtoinaccurateassumptionsaboutthequalityofstudentlearningortheeffectivenessofschoolsandteachersandcanbringaboutinterventionsthatmaynotbenecessary.InFigure15.2welistsomeofthecharacteristicsofahigh-quality,coherentassessmentsys-tem.Butwhatwouldacoherentsystemlooklike?Table15.1,page312,isaframeworkforanassessmentsystemadaptedfromtheonedevelopedbythestateofMainein2003(NRC2003).Weincludeitheretoillustratehowmultiple assessments canbe incorporatedcoherently intoa singleassessment system.6

_______6Thesystemhassincebeenmodifiedandsothedescriptionincludedheredoesnotde-

scribethecurrent(2007)Maineassessmentsystem.




Figure 15.2 characteristics of a High-Quality Science Assessment System

Thefollowingarecharacteristicsofanassessmentsystemthatcouldprovidevalidandreliableinformationtothemultiplelevelsoftheeducationsystemandcouldsupporttheongoingdevelopmentofstudents’scienceunderstanding:

1. incorporatesassessmentsthatarecloselyalignedtothestandardsthatguidethesystemandisstructuredsothatallelementsarecoherentwiththegoals,curriculummaterials,andinstructionalstrategiesofthescienceeducationsystemofwhichitisapart;

2. includesarangeofmeasurementapproachesandmultiplemeasuresofachievementthatprovideavarietyofevidencetosupporteducationaldecision-makingatdifferentlevelsofthesystem;

3.containsmeasuresthatassessstudentprogressovertimeratherthanrelyingsolelyonone-time,large-scaletestingopportunities;

4. isusefulinthesensethattheassessmentresultsaremadeaccessibleandarereportedinatimelymannertothosewhoneedthem;

5. fitsintoalargereducationsystemthatprovidesthenecessaryresourcesforthedevelopment,operation,andcontinuedimprovementofboththeassessmentsystemandtheeducationsystemwhenassessmentresultsindicateimprovementisnecessary;

6.providessystematic,ongoingprofessionaldevelopmentforteachersandothersoncurrentscienceassessmentpractices,theusesandlimitationsofassessmentresults,andprocessesfordevelopingandusingsoundassessments;and

7. issystemicallyvalid—thatis,itpromotesintheeducationsystemdesiredcurricularandinstructionalchangesthatresultinincreasedlearningandnotjustimprovementintestscores.

Source: National research council (Nrc). 2006. Assessment in support of instruction and learning: Bridging the gap between large-scale and classroom assessment. Washington, Dc: National Academy Press, p. 28.




TheassessmentsysteminTable15.1wasdesignedtomeetthreeprinci-palgoals:(1)providehigh-qualityinformationaboutstudentperformancetoinformteachingandlearning,(2)monitorschoolsandadministrativeunitsandholdthemaccountablefortheirsuccessatmakingsurestudentsmeetthestatestandards,and(3)certify thatstudentshavemetthestatestandards.

A Developmental Approach to AssessmentInkeepingwiththecommittee’sconclusionthatscienceeducationandas-sessmentshouldbebasedonafoundationofhowastudent’sunderstandingofsciencedevelopsovertimewithcompetentinstruction,thecommitteecalled for a developmental approach to science assessment (see MastersandForster1996,1997).Adevelopmentalapproachmeansgatheringevi-denceofthedevelopmentofstudents’understandingovertime,asopposed

table 15.1 Framework for an Assessment System

Primarypurposeoftheassessment

Whoselectsordevelopstheassessment?

Whoscorestheassessment?

ClassroomAssessment

Informingteaching and learning

Individualteacher Individualteacher

SchoolorDistrictAssessment

Informingandmonitoring

Groupsofteachersand administrators

Groupsofteachers(andothers)

State Assessment Monitoringandevaluatingprogramstoensureaccountability

Groupsofadministrators, administrators, and/orpolicy-makers

Scorersoutsidethe district

Assessment System

Informingteaching, monitoring andevaluating,certification

Districtassessment leadership

Bothinternalandexternal




toonlyataspecificpointintime.Thisapproachrecognizesthatsciencelearningisnotsimplyaprocessofacquiringmoreknowledgeandskills,butratheraprocessofprogressingtowardgreaterlevelsofcompetenceandun-derstandingasnewknowledgeislinkedtoexistingknowledgeandasnewunderstandingsbuildonandreplaceearlierconceptions. Ahallmarkofadevelopmentalapproachistheuseofmultipleassess-mentsthatcollectivelyprovideinformationaboutstudents’learningovertime.Students’performancesonthesemultipleassessmentsarecomparedagainst a pre-constructed progress map or learning progression that de-scribeswhatstudentsareexpectedto learnandhowthat learning isex-pectedtounfoldasthestudentprogressesthroughtheinstructionalmate-rial.Recordsaremaintainedaboutperformanceoneachassessmentandarecombinedandviewedasawholetoestimatestudents’levelofachievementinaparticularareaoflearning.Theestimateofastudent’scurrentlocationontheprogressmaporlearningprogressionservesasaguidetothekindsoflearningexperiencesthatwouldhelpthemdeveloptheknowledge,under-standing,andskillsnecessarytoprogress.

Professional Development and AssessmentRegardlessoftheinformationitisdesignedtocollect,assessment,onitsown,cannot improve student learning—it is the way in which assessment andtheresultsareusedthatcanaccomplishthatgoal.Foranassessmentprocesstofunctionas itshould,everyonewhousesassessmentresultsneedstobeassessmentliterate.Thatis,theyneedtohaveaclearunderstandingofthepurposesandlimitationsofassessment,goalsandpurposesforassessments,thewaysinwhichdifferentassessmentsfunction,andhowtointerpretanduseassessmentresultsappropriately.Thosewhoneedtodeveloptheirassess-mentliteracyincludeseveryonefromelectedofficialsatthehighestlevelstoschoolboardmemberstoparentstostudentstoteachers.Theteachersneedtodeveloptheirknowledgeofassessmentthemostbecausetheyareinthebestpositiontousetheresultstoimprovestudentlearning. Althoughteachersmaynotbeinvolvedindesigningtheirstate’stests,theyshouldunderstandthekindsofdatathatareproducedbythesetestsandhowthe resultscanandwillbeused.Further they shouldhave suf-ficientunderstandingofthetechnicalpropertiesoftheassessmentstoputtheresultsincontextandtolinkthemtootherpiecesofinformationtheyhaveabouttheirstudents.




Assessmentactivitiesrequirethatteachershave,inadditiontoadeepunderstandingofthecontentdomainstheyareteaching,knowledgeabouthowtodeveloptasksthatarevalidandusefulintheclassroomandhowtousetheresultsoftheseassessmentsininstructionallysupportiveways.Theymustalsounderstandtheusesandlimitationsofexternalassessment,suchaslarge-scalestatewidetests,andbecognizantofthewaysinwhichsuchassessmentaffectstheirteaching.Figure15.3isalistofsomeofthethingsthatassessment-literateteachersshouldknowandbeabletodo.

Figure 15.3 Assessment competencies for Teachers

Teachersshouldbeableto

1. chooseassessmentmethodsappropriateforinstructionaldecisions,2. developassessmentmethodsappropriateforinstructionaldecisions,3. administer, score,andinterprettheresultsofbothexternallypro-

ducedandteacher-producedassessmentmethods,4. useassessmentresultswhenmakingdecisionsaboutindividualstu-

dents, planning teaching, developing curriculum, and school im-provement,

5. developvalidpupilgradingproceduresthatusepupilassessments,6. communicateassessmentresultstostudents,parents,otherlayaudi-

ences,andothereducators,and7. recognizeunethical,illegal,andotherwiseinappropriateassessment

methodsandusesofassessmentinformation.

Source: American Federation of Teachers, National council on measurement in education, and National education Association. Standards for teacher competence in educational assessment of students. Available at www.lib.muohio.edu/edpsych/stevens_stand.pdf

Thecommitteefoundthatinserviceteachersneedtohavetheopportu-nity,andtaketheresponsibility,todeveloptheirassessmentliteracy.Tak-ing classes, participating in professional development opportunities, and




keepingupwithissuessurroundingassessmentaresomeofthewaysthatteacherscandeveloptheirownassessmentliteracy.Byorganizingandpar-ticipatinginopportunitiestodiscussstudentworkthatresultsfromassess-ment,teacherscandevelopacommonunderstandingofwhatcompetenceinadomainlookslikeandhowtosupportstudentprogresstowardachiev-ing it.

conclusionsDesigninghigh-quality scienceassessment is an importantbutdifficultgoal to achieve. Science assessment must target the knowledge, skills,and habits of mind that are necessary for science literacy, and it mustreflectcurrentscientificknowledgeandunderstandinginwaysthatarescientificallyaccurateandconsistentwith theways inwhich scientistsunderstandtheworld.Itmustassessunderstandingofscienceasacontentdomainandanunderstandingofscienceasaspecificwayofthinking.Itmustalsoprovideevidencethatstudentscanapplytheirknowledgeap-propriatelyandthat theyarebuildingontheirexistingknowledgeandskillsinwaysthatwillleadtomorecompleteunderstandingofthekeyprinciplesandbigideasofscience.Addingtothechallenge,competenceinscienceismultifacetedanddoesnotfollowasingularpath.Scienceas-sessmentmustaddressthesecomplexitieswhilealsomeetingprofessionaltechnicalstandardsforreliability,validity,andfairnessforthepurposesforwhichtheresultswillbeused. The committee concluded that the goal of developing high-qualityscienceassessmentwillonlybeachievedthoughthecombinedeffortsofscientists, science educators, developmental and cognitive psychologists,expertsonlearning,lawandpolicymakers,andeducationalmeasurementspecialistsworkingcollaborativelyratherthanseparately.

referencesAmericanAssociationfortheAdvancementofScience(AAAS).1993.Benchmarks for

science literacy. NewYork:OxfordUniversityPress.

AmericanEducationResearchAssociation(AERA).2003.Research points: Standards and tests, keeping them aligned, L.B.Resnick,ed.Spring(1,1).Washington,DC:AERA.

Masters,G.,andM.Forster.(1996,1997)Developmental assessment: Assessment resource kit.Hawthorne,Australia:AustralianCouncilonEducationalResearchPress.

NationalResearchCouncil(NRC).1996.National science education standards. National Committee on Science Education Standards and Assessment. Center for Science,




Mathematics, and Engineering Education. Washington, DC: National AcademyPress.

National Research Council (NRC). 2000. Testing English-language learners: Report and workshop summary. BoardonTestingandAssessment,CommitteeonEducationalEx-cellenceandEquity.Washington,DC:NationalAcademyPress.

NationalResearchCouncil(NRC).2003.Assessment in support of instruction and learning: Bridging the gap between large-scale and classroom assessment.BoardonTestingandAs-sessment,CommitteeonScienceEducationK–12, and theMathematicalSciencesEducationBoard.WashingtonDC:NationalAcademyPress.

NationalResearchCouncil(NRC).2004.Keeping score for all: The effects of inclusion and accommodation policies on large-scale educational assessments. CommitteeonParticipa-tionofEnglishLanguageLearnersandStudentswithDisabilitiesinNAEPandOtherLarge-ScaleAssessments.Washington,DC:NationalAcademiesPress.

NationalResearchCouncil(NRC).2006.Systems for state science assessment.CommitteeonTestDesignforK–12ScienceAchievement.M.R.WilsonandM.W.Bertenthal,eds.BoardonTestingandAssessment,CenterforEducation,DivisionofBehavioralandSocialSciencesandEducation.Washington,DC:NationalAcademiesPress.

Perkins,D.1998.Whatisunderstanding? InM.S.Wiske(Ed.),Teaching for understanding: Linking research with practice. SanFrancisco:Jossey-Bass.

Reiser,B.J.,J.Krajcik,E.Moje,andR.Marx.2003.Designstrategiesfordevelopingsci-enceinstructionalmaterials.PaperpresentedattheNationalAssociationforResearchinScienceTeachingmeeting,Philadelphia(March).

Smith,C.,M.Wiser,A.Anderson,andJ.Krajcik.2006.(FocusArticleofcombineddou-bleissueofjournal):Implicationsofresearchonchildren’slearningforstandardsandassessment:Aproposedlearningprogressionformatterandatomic-moleculartheory.Measurement14(1&2):1–98.

Wiggins,G.P.,andJ.McTighe.1998.Understanding by design.Alexandria,VA:Associa-tionforSupervisionandCurriculumDevelopment.

Wilson,M.2005.Constructing measures: An item response modeling approach.Mahwah,NJ:LawrenceErlbaum.


Documents

chapter 15 Systems for State Science Assessment: …...Assessment: Findings of the National research council’s committee on Test Design for k–12 Science Achievement Meryl W. Bertenthal