MLOPMeasurableLearningObjectivesProject

SuzanneWhiteBrahmiaUniversityofWashington

Collaborators

• RutgersUniversity– Chaz Ruggieri– EugeniaEtkina

• CornellUniversity– NatashaHolmes– EmilySmith

Purpose

• Focusoncoursetransformation• Exampleofimplementation• Sampleofoutcomes

Env

iron

men

ts I. Disseminating:CURRICULUM & PEDAGOGY Change Process: Tell/Teach individuals about new teaching conceptions and/or practices and encourage their use.

Diffusion & Implementation

II. Developing: REFLECTIVE TEACHERS Change Process: Encourage/Support individuals to develop new teaching conceptions and/or practices.

Scholarly Teaching & Faculty Learning Communities

Indi

vidu

als

III. Enacting:POLICY Change Process:Require/Encourage new teaching conceptions and/or practices.

Organizational Development

IV. Developing:SHARED VISION

Change Process: Empower/Support stakeholders to collectively develop new environmental features that encourage new teaching practices.

Learning Organizations

InstitutionalChangeStrategiesHendersonetal.(2011)

Prescribed Emergent

Env

iron

men

ts I. Disseminating:CURRICULUM & PEDAGOGY Change Process: Tell/Teach individuals about new teaching conceptions and/or practices and encourage their use.

Diffusion & Implementation

II. Developing: REFLECTIVE TEACHERS Change Process: Encourage/Support individuals to develop new teaching conceptions and/or practices.

Scholarly Teaching & Faculty Learning Communities

Indi

vidu

als

III. Enacting:POLICY Change Process:Require/Encourage new teaching conceptions and/or practices.

Organizational Development

IV. Developing:SHARED VISION

Change Process: Empower/Support stakeholders to collectively develop new environmental features that encourage new teaching practices.

Learning Organizations

InstitutionalChangeStrategiesHendersonetal.(2011)

Prescribed Emergent

Buildingonpriorwork

• ExistingLOsthathavegonethroughmultipleiterationsw/varietyoffaculty:mostlyfromUCBoulderandUBCSEIs

• Strongforsomeupperdivisionphysicscourses– broadfacultyinterest/creativeeffort

FacultycollaborationcreatingLOs(LearningObjectives)

(Chasteen etal.2011)

• Focusonemergentintendedoutcomesratherthanprescribedintendedoutcomes

• Giveslanguageandvalidatesideasofbroadfacultygroup

• Resonanceofinstructionaleffort• Helpstogenerateasharedvision– catalyzecommunicationandunderstanding– provideaframeworkforsustainingpedagogicalandculturalchange

UncharteredTerritory

• Consideringtheintroductorycoursesaspartofacoherentsequence,primarilyfornon-majorsSequencelevelobjectives

• Consideringmathematizationasanin-courseandacross-courseprogressionSequencelevelmappingofmathematization

• ExploringmethodsofmeasuringwhetherobjectiveshavebeenmetPragmaticanditerativedevelopmentofLOs

IntroductoryPhysicsMeasurableLearningObjectives

• Consideringtheintroductorycoursesaspartofacoherentsequence,primarilyfornon-majorsSequencelevelobjectives

• Consideringmathematizationasanin-courseandacross-courseprogressionSequencelevelmappingofmathematization

• ExploringmethodsofmeasuringwhetherobjectiveshavebeenmetPragmaticanditerativedevelopmentofLOs

IntroductoryPhysicsMeasurableLearningObjectives

• Consideringtheintroductorycoursesaspartofacoherentsequence,primarilyfornon-majorsSequencelevelobjectives

• Consideringmathematizationasanin-courseandacross-courseprogressionSequencelevelmappingofmathematization

• ExploringmethodsofmeasuringwhetherobjectiveshavebeenmetPragmaticanditerativedevelopmentofLOs

StructureofLOs

Source of current learning objectives (LOs)Source Intellectual Merit

NGSS Science Practices Carefully crafted; breadth of education and physicist expertiseAP Physics Practices – College Board

SEI Learning Objectives (UC Boulder and UBC)

PER developed or contributed, evidence-based objectives, measures

and recommendations

Scientific Abilities Rubrics (Rutgers) SCALE-UP (NCSU) AAPT Recommendations for Undergraduate Laboratory Curriculum 11

AAPT Report: Supporting LGBT+ Physicists and Best Practices Guide for Departments 4

ABET (Accreditation Board for Engineering and Technology) criteria

Accredits post-secondary education programs in "applied science, computing, engineering, and

engineering technology"

ATC21S – Assessment & Teaching of 21st century skills. (2009).

Study of the effectiveness of K-16 education at preparing students for the

STEM workplace (sponsored by Microsoft, Intel, Cisco)

SequenceLevelLOs

Threemaincategories:• HM:– PhysicsHabitsofMind

• ML:– Understandingmodelsandtheirlimitsinphysics

• PW:– ProfessionalismandWorkplace

HM:PhysicsHabitsofMind① TranslationBetweenPhysicalandSymbolicWorld:Developtheinclinationand

abilitytotranslatebetweenthephysicalandsymbolicworldsinanefforttoquantitativelyreasonabouthownatureworks.

② Reasoningwithphysicalquantities:Reasonabstractlyandquantitativelywithnewscalarandvectorquantities:makephysicalsenseofthequantitiesandmasteringtheirmathematicalstructures.

③ MultipleRepresentations:Createandtranslatebetweenmultiplerepresentationsofthesameconcept(eg,text,equations,graphs,diagrams).

④ ProblemArticulation:Articulatewhatitisthatneedstobesolvedinaparticularproblem,whatisknownandrepresentthemusinganon-verbalrepresentation.

⑤ Perseverance:Recognizethatwrongturnsarevaluableinlearningthematerial,recoverfrommistakes,andpersistinginworkingtothesolutionevenwhenthereisnoclearpathtotheendpoint.

⑥ Sensemakingwithquantity:Effectivelyuseunitreasoning,vectorandscalarnaturesandlimitingcasestomakesenseofanswers.

⑦ OrderofMagnitudeandReasonableness:Anticipatetheorderofmagnitudetojudgethereasonablenessofmeasurementsandcalculations.

⑧ Reasoningbasedonmathematicalstructure:Lookforandmakeuseofpatternsassociatedwithmathematicalstructuretoreasonacrosscontextsandscale.

⑨ RecognizingUncertainties:Beabletorecognizethatallmeasuredquantitieshaveinherentuncertainties.

① Makingobservations:Formascientificquestion,designandcarryoutexperimentstolookforpatterns.

② Developingamodel:analyzeandinterpretdatawhileattendingtouncertaintyinmeasurementandconstructexplanationsbasedonpatternsinthedata.

③ Reasoningwithmathematicalmodels:Developandusemathematicalmodelsandexplanations,constructviablearguments,engageinargumentationfromevidenceandcritiquereasoningofothers.

④ Modellimitation: Articulateassumptionsmadewhenapplyingamodel,andtherangeoverwhichaparticularmodelisavaliddescriptionofnature.

⑤ Modeltesting: Designanexperimenttotestthemodelandmakeapredictionoftheoutcomebasedonit.

⑥ Scientificjudgment:analyzeandinterpretdatafromatestingexperimentwhileattendingtouncertainty,andmakeascientificjudgmentabouttheoutcome.

ML:Understandingmodelsandtheirlimitsinphysics

PW:ProfessionalismandWorkplace① CollectiveIntelligence:Recognizingthetwofeaturesofhighcollective

intelligence,andmonitoringsocialclimatetooptimizethesefeatures(equitablespeakingturns,socialsensitivity).

② Collaboration:Abletoarticulateaffordancesthatagroupbringstoarrivingatacreativesolution,fulfillstherolesthatmakeupeffectivegroups

③ Inclusion:Demonstrateeffectivecommunicationskillsinthecontextofarecitationorlabgroupthatresultsinwhole-groupmeaningfulparticipation.

④ Communicatingphysics:Beabletocommunicatephysicsinwrittenandoralforms.

⑤ IndependentLearning: Recognizingandactingonconfusion:Beabletoarticulatespecificallythesourceofconfusionandtakingactiontomovebeyondthatdifficulty(e.g.,officehours,groupstudy)

⑥ SkepticismTowardConclusions:Recognizethatscientificconclusions–whetherfromanoutsidesourceorfromyourowncalculations– maybeincorrect,anddeveloptheabilitytochecktheseconclusionswithsimplecalculations,3rdpartyinformation,and/orcommonsense.

IntroductoryPhysicsMeasurableLearningObjectives

• Consideringtheintroductorycoursesaspartofacoherentsequence,primarilyfornon-majorsSequencelevelobjectives

• Consideringmathematizationasanin-courseandacross-courseprogressionSequencelevelmappingofmathematization

• ExploringmethodsofmeasuringwhetherobjectiveshavebeenmetPragmaticanditerativedevelopmentofLOs

MathematizationMatrix

• Mapsacross25topicsthatmakeupintroductoryphysics– 8mathematicalhabitsofmind– ~60mathematicalobjects

• Underdevelopment– canbeusedtosetMLOsandasatoolforsummativeassessment

MathematizationMatrix:Excerpt

IntroductoryPhysicsMeasurableLearningObjectives

• Consideringtheintroductorycoursesaspartofacoherentsequence,primarilyfornon-majorsSequencelevelobjectives

• Consideringmathematizationasanin-courseandacross-courseprogressionSequencelevelmappingofmathematization

• ExploringmethodsofmeasuringwhetherobjectiveshavebeenmetPragmaticanditerativedevelopmentofLOs

Coursetransformationasaniterativeprocess

RefinementofLOs

Developmentofcoursecontentandstructure

AssessmentandReflection

Pilot:Calculus-basedE&McourseatRutgers• Threemajorstructuralchangeswere– coursecontentdecisionreachedbyconsensus– collaborativegroupactivitiesreplacedinstructor-at-the-boardproblemsolvinginrecitation

– couplingofthelabandlecturecourse• Courseleadership:– threeseniorfacultymemberswhoextendedintoa4thyeartoparticipateinimprovingthecourse;

– 2stafflaboratorymanagementteam• Recitation/labmaterialswereinspiredbyordrawnfromresearch-validatedsources,andlocallyadaptedtoreachconsensus

MLOsasCatalysttosustainedtransformationin2nd year

• priortotransformation– lecturedroveallcoursecontent

• typicalchangeoflecturereveryfewyearsleftcoursesdisjointed

• consensusaroundMLOshighlightedcentralrolelabactivitiesplayinmeetingMLOs

• Evidenceofeffectivenesscoupledtotheconsensus-drivenMLOs– sustainedtransformation(broaddepartmentalsupport)

Mappingcourseactivitiesforthe1stweekofAnalyticalPhysicsIIA to

SequenceLevelLOs.

Mappingcourseactivitiesforthe1stweekofAnalyticalPhysicsIIA to

SequenceLevelLOs.

FormativeAssessment:ScientificAbilityRubrics

SummativeAssessment:Historiccomparisons

• E-CLASS(ColoradoLearningAttitudesaboutScienceforExperimentalPhysics) Zwickl etal.2014– E-CLASSassessesstudentsviewsabouttheirstrategies,habitsofmind,andattitudeswhendoingexperimentsinlabclasses.

• Studentcourseevaluations:– numberandnatureoffreeresponse– Averagesonstandardlikert-scaleassessmentitems

E-CLASSHistoricComparison

SARubricsandE-CLASSasmeasuresofSequenceLOs

Whatdoyoulikebestaboutthiscourse?

Nearfutureresearchactivity• Mappingoutthemathematizationmatrix• Qualitativestudy(Rutgers,Cornell,UW)understandingfacultyperceptionsofvalueandinfluence

• Investigatingformativeandsummativemethodsofassessingwhethergoalshavebeenmet(inlab,lecture,andrecitationportionsofcourse)

• impact ofanMLOP-centeredcourseonstudentlearningandlearningculture

• PhysPort:DesigningwebinterfaceforeasynavigationthroughandselectionofpersonalizedLOs

Relatedpresentationsatthismeeting

• MoredetailsabouttheprocessofdevelopingsharedvisioninthecontextofE&M

• CharlesRuggieri• Newlaboratoryassessmentinstrumentalignedwithcriticalthinkinglaboratorylearningobjectives

• KatherineQuinnNatasha’sstudentPLIC• Moredetailonrestuls ofimplementationinthecontextofacourse+lab transformations

• DebbieAndres&Chaz Ruggieri poster

Electric/MagneticForce:ApplyCoulomb’sLaw,Biot Savart andAmpere’slawandthesuperpositionprincipletofindthenetforceduetoadistributionofcharges/currents.Electric/MagneticFields:ArticulatewaysthatscalarandvectorfieldsexistinspaceandcanbeusedtoexplaininteractionsElectricPotentialEnergy:Compareandcontrasttheelectricpotentialenergywithgravitationalpotentialenergy.DemonstrateunderstandingofElectricPotential:GiveaconceptualdescriptionofVanditsrelationshiptoenergyanddescribetherelationshipbetweenVandtheEMFinacircuit.Maxwell’sEquations:ArticulateMaxwell’sequationsinwords,andarguethattheyareacompletemathematicaldescriptionofelectricandmagneticphenomenaandtherelationshipbetweenthetwo.SophisticatedQuantitiesinE&M: SpontaneouslyreasonwithvectorandscalararithmeticappropriatelyinthecontextofEMquantities,andunderstandtheroleof+/- signs,SophisticatedCalculationsinE&M: Exploitsymmetriestoperformcalculationsthatinvolvepathandsurfaceintegrals.SophisticatedRepresentationsinE&M: Representvectorandscalarfieldsusingfieldlinesandequipotentiallines,createanddescribespatialgraphsoffieldandpotential,andtemporalgraphsofcurrent,voltageandcharge,Poynting vectors,spatialgraphsofEMwaves

CourseLevelLOsExampleIntroE&M:TheoryandReasoning

Electric/MagneticFieldCalculations:CalculatetheelectricfieldusingCoulomb’slaw,usingGauss’slaw,andfromtheelectricpotential,andmagneticfieldfromAmpere’slaw.

Materialproperties: Describetheinteractionofmatterwithelectricfields,magneticfieldsandEMwaves

Circuits: Demonstratethroughwordsandcalculationstheconservationofchargeandenergyinthecontextofelectriccircuits,andmodelthetimedependenceofthechargeonacapacitor,andthecurrentthrougharesistorinanRLandLCcircuit.

Circuitmeasurementsandcalculations: Constructsimplecircuits,demonstrateabilitytomeasurecurrentandvoltageandusingamultimeter,calculateequivalentcapacitanceandresistancefrommeasuredquantities

ApplicationofMaxwell’sEquations: ExplainthephysicsbehindhowanACtoACtransformerworks.

CourseLevelLOsExampleIntroE&M:Application

Situatingthisproject

• Expandsonearlierwork• EstablishescoherentsetofMLOPacrosstheintroductorysequence

• Alignswith(sitecurrentreportshere)• Designedto:– Provideafoundation,notaprescription– Editableandmodulartofacilitatelocaladoptionandemphases

– Providemethodologyforassessment,orindicateareasofneedinthisregardifnothingexists

UnitLevelLOsExampleIntroE&M:

Pilot• Calculus-basedE&McourseatRutgers• Threemajorstructuralchangeswere

– coursecontentdecisionreachedbyconsensus– collaborativegroupactivitiesreplacedinstructor-at-the-boardproblem

solvinginrecitation– couplingofthelabandlecturecourse

• Courseleadership:– threeseniorfacultymemberswhoextendedintoa4th yearto

participateinimprovingthecourse;– laboratorymanagementteamoftwostaffmembers

• Recitationactivitiesdevelopedinconjunction/byconsensuswithPERfaculty

• Materialsweredrawnfrom:– UWTutorials,– *ActiveLearningGuide– ISLElabs– locallydeveloped

*ancillaryofCollegePhysics,Etkina,GentileandVanHeuvelen

InstructionalChangeStrategiesFairweather (2009)

ImprovingTeaching:• Identifyeffectiveresearch-basedinstructionstrategiestoimprovestudent

learning• Improvementbasedonagreementbetweeninstructorpracticeandtarget

practice(e.g.clickers)

Improvingstudentlearning:• Defineslearninggoalsanddeviseswaystomeasurebasedonthosegoals.• Improvementbasedonimprovementinmeasuresofstudentlearning.

Improvingstudentproductivity:• Identifiesleastexpensive/easiestwaytoattaingreatestincreaseinstudent

learningacrossprogram• Improvementoftenbasedontheeliminationofleasteffectivepractices(e.g.

goingfromtraditionaltocollaborativerecitationsections.)

InstructionalChangeStrategiesFairweather (2009)

ImprovingTeaching:• Identifyeffectiveresearch-basedinstructionstrategiestoimprovestudent

learning• Improvementbasedonagreementbetweeninstructorpracticeandtarget

practice(e.g.clickers)

Improvingstudentlearning:• Defineslearninggoalsanddeviseswaystomeasurebasedonthosegoals.• Improvementbasedonimprovementinmeasuresofstudentlearning.

Improvingstudentproductivity:• Identifiesleastexpensive/easiestwaytoattaingreatestincreaseinstudent

learningacrossprogram• Improvementoftenbasedontheeliminationofleasteffectivepractices(e.g.

goingfromtraditionaltocollaborativerecitationsections.)

InstitutionalChangeStrategies