Appendix A – Fact-Finding Protocols · 2016. 5. 30. · been a significant part of curriculum development for over sixty years. Tyler (1949) proposed a model of curriculum design

SenateWorkingGrouponLearningOutcomesAppendices

1

AppendixA–Fact-FindingProtocolsAcademicUnitQuestions

a. HowdoesyourfacultymanagetheLOdevelopmentprocess?a. Whatistherecommendedprocess/workflowforlearningoutcomedevelopment?b. Whodoyouconsultwithandinwhatorder?c. Whatresourcesareincommonused. Doesthisincludealignmentofassessment?e. Whointhefacultyhasresponsibilityforoversight?f. Howoftendoprogramsre-evaluatetheirlearningoutcomes,andwhattriggersthat?

b. WhatarethepolicyrequirementsrelatedtoLOs inthefaculty(e.g.FAHSSsyllabusregulations,OdetteAssuranceofLearningprocesses)?

c. Whatisyourexperienceoftheperceivedvalueoflearningoutcomesindepartments/theFaculty,andforwhatpurposesaretheyused?

d. Whatexamplesofparticularlyeffectivepracticecanwe identify (e.g.curriculummaps,syllabustemplates)?

e. Existingstrengths,areasfordevelopment,andbottlenecks?f. Whatcausesfrustration/burnout?Whathashelped?g. Whatsuggestionsdoyouhaveforimprovementstothecurrentprocess?h. Documentcollection–policies,templates,samplesyllabi,otherresources?LearningOutcomesAssessmenta. Howareprogramlearningoutcomesbeingassessed?b. Whatpractices andprocesses are inplace for systematically gatheringprogrammatic learning

outcomeassessmentinformation?c. Howarelearningoutcomesassessmentdatabeingused?

Is there anyone else in your department/the faculty that you suggest I speak to because of theirinvolvementwithlearningoutcomesprocessesintheunit?

CentralSupportUnits

a. What is the unit’s contribution/role in institutional requirements for learning outcomedevelopment(QualityCouncil/IQAP,MTCU,newprograms,cyclicalreview)?Whocomestoyou,what do they bring you, what do you do for them, who do you report to, and who do youcollaboratewith?

b. Whatprocessesandresources(includingnumberofpeople)areinvolvedinmeetingtheunit’sresponsibilitiesforlearningoutcomedevelopment?Howissupportorganized?

c. Whatpoliciesorguidelinesrelatedtolearningoutcomesaremanagedthroughthisunit?d. Whataretheboundariesofwhatyouareresponsiblefor(forexample,wesupportnewprogram

development,butnotnewcoursedevelopment)?e. Existingstrengths,areasfordevelopment,andbottlenecksf. Whatcauses frustration/burnout (foryourunitor for thoseyousupport)?Whathasorwould

help?g. Whatsuggestionsdoyouhaveforimprovementstothecurrentprocess?h. Documentcollection–policies,templates,syllabi,otherresources

Sa160610-5.2.2Appendices


2

AppendixB:LearningOutcomes:BriefHistoryandInternationalContext

Manyoftheessentialideasconnectedwithlearning-outcomesbasedapproachestoeducationhavebeenasignificantpartofcurriculumdevelopmentforoversixtyyears.Tyler(1949)proposedamodelofcurriculumdesignbasedontheestablishmentofcurricularobjectivesexpressedintermsofthekindsofbehaviourthelearnershoulddevelop,andthecontextinwhichthatbehaviourisintendedtooperate.Animportantpremiseofhisworkisitsemphasisonlearningratherthanteaching:students’activebehavioursaslearners,nottheteachers’activitiesarethecorefociofcurriculumdesign.Sincetheearly1980s,Biggsisoftencreditedforclarifyingthekeyideasofoutcome-basededucation,addingrigourtotheirlogic,systematizingtheapproachestheyimply,andconnectingthemwithconstructivistandphenomenologicalapproachestolearning–atleastinthecontextofhighereducation.InrecentdecadesBiggs’snotionofconstructivealignment(1996,1999)hasprovidedaclearoperationalbasisforrefiningandcreatingintendedlearningoutcomesandworkingbackwardsfromthemtocreateaproductivelearningenvironmentthroughteaching/learningactivitiesthatwouldengagestudentsinactivitiesclearlyalignedwithachievingintendedoutcomes,andassessmenttasksthatvalidlyjudgethedegreetowhichstudentshaveachievedthem.Theimplementationofthisparticularlearning-outcomes-basedapproach,constructivealignment,canplayanimportantroleinsystematicallyshiftingpedagogicalresourcestowardlearning(Spronken-Smithetal.,2013).Inliteratureonpost-secondaryteachingandlearning,BarrandTagg’snowfoundationalmanifestoonshiftingfromateaching-centredtolearning-centredparadigmechoesBiggs’principles(1995),whileaddingfurtherconsiderationsforpost-secondaryeducators.

However,formalrequirementsforlearningoutcomesdevelopmentinourcurrentcontextarealsorelatedtoother,morebureaucraticconcernsandpolicyprioritiesinthepost-secondarysectorinOntario.Amongtheseotheruses,Deller,Brumwell&MacFarlane(2015),forexample,recommendusinglearningoutcomestoassesswhetherorstudentsarelearningwhatinstitutionsclaimtheyare,whetherinstitutionsareidentifyingandimplementingteachingpracticesthateffectivelydevelopstudentskills,andtofacilitatecreationofa“commonlanguagesothatmembersofthePSEsector,thelabourmarketandthepubliccandiscusswhatisexpectedofpostsecondarygraduates”(p.2).

Thedevelopmentandassessmentofprogrammaticlearningoutcomesasalsobeenco-optedasanelementofthegrowingemphasisonqualityassessment,studentmobilityandcredentialharmonizationglobally(Altbach,Reisberg&Rumbley,2009).TheBolognaProcess,originallyratifiedin1999,hasbeenakeydriverinthistrend.Itinitiallysoughttoharmonizeandclarifypost-secondarycredentialsacrossEUcountries-withgoalsincludinggreaterinternationalmobilityandenhancedglobalcompetitivenessinthepost-secondarysector(Wihlborgh&Teelken,2014)-andinfluencedthedevelopmentofcomparableeffortsintheUSandLatinAmerica(Crosier&Parveva,2013).Itsignificantlyexpandeddialogue,awarenessandactivityrelatedtothedevelopmentoflearningoutcomesandotherpracticesintendedtoimprovethetransparencyandclarityofstudentachievementindifferentinstitutionsandprogramsinternationally.Similareffortsinothercountries,suchasthedevelopmentofLearningandTeachingAcademicStandardsin2010inAustralia,andtheestablishmentofSubjectBenchmarkStatementsbytheUKQualityAssuranceAgency,havefocusedondiscipline-specificlearningoutcomes.


3

TheOntariocontext

Ontariouniversitiesoperateunderaqualityassurancesystemthatinvolveslearningoutcomes,degree-levelexpectations(DLEs)andaqualityassuranceframework.DLEsserveasOntariouniversities’academicstandards,andidentifytheknowledgeandskilloutcomecompetenciesthatreflectprogressivelevelsofintellectualandcreativedevelopmentassociatedwithvariouslevelsofdegrees(OntarioQualityAssuranceFramework(OQAF,2010).DLEsfunctionasageneralframeworkoutliningwhatstudentsshouldknowandbeabletodoaftersuccessfulcompletionofadegreeprogramofanylevel:inotherwords;theyareintendedtoarticulatehowdegreecredentialscomparetothoseinotherjurisdictions,andtoprovideabasicframeworkforcurriculumplanningatuniversities(COU,2011).EachuniversitythencreatesalessgeneralsetofDLEsfortheirowninstitutionaluse,reflectingitsownmission,ethos,valuesandculture–attheUniversityofWindsorthesearecalled“GraduateAttributes”

Eachprogramateachinstitutionisstructuredbasedonintendedprogram-levellearningoutcomesthatdescribewhatastudentgraduatingfromaspecificprogramshouldbeabletodo–theexpectedachievementsofallsuccessfulgraduatesofthatprogram.Thesetofprogram-levellearningoutcomesmustmapinsomewaytoalloftheUniversityofWindsorgraduateattributes,whichinturnmapontotheprovincialdegree-levelexpectations.Eachcoursemustincludecourse-levellearningoutcomesthatmapontoasmallersetofthegraduateattributes.Collectivelysuccessfulachievementofallcourseandprogram-levellearningoutcomeswithinaprogrammustenablestudentstoacquirealloftheUWindsorGraduateAttributesbythetimetheygraduate.Thefocusoflearningoutcomesisonthelearner,andtheoutcomesshouldformthebasisofhowstudentsareassessedandevaluated.

Thiscommitmentwasformalizedthroughthe2010establishmentoftheOntarioCouncilofUniversitiesQualityCouncil(QC)QualityAssuranceFramework.Theexpectationisthatallprogramsandallcourseswithinprogramswillcontainintendedlearningoutcomes.TheseformacomponentofqualityassurancereviewthroughtheQC,andprogram-levellearningoutcomesmayalsoplayaroleintheevaluationofprogramdistinctivenessandStrategicMandateAgreement(SMA)alignmentinnewprogramevaluationbytheMinistryofTraining,CultureandUniversities(MTCU).Throughinstitutionallydevelopedqualityassuranceprotocolsbasedonthisframework,allOntariouniversitiessubmitprogram-andcourse-levellearningoutcomesaspartofcyclicalexternalreviewofexistingprogram,andasnewprogramsaredeveloped.ThisprocessisoverseenbytheOntarioUniversitiesCouncilonQualityAssurance.

In2013,theMTCUpublishedOntario’sDifferentiationPolicyFrameworkforPostsecondaryEducation,whichidentifieddifferentiationas“aprimarypolicydriverforthesystem”,intendedtobuildonandfocusthewell-establishedstrengthsofinstitutionsandtohelpthemtooperateinacomplementaryfashion.Studentlearningoutcomesareidentifiedasapotentialareaformetricsidentifyinguniquestrengths.TheMinistry’srecentconsultationreport,FocusonOutcomes,CentreonStudents:PerspectivesonEvolvingOntario’sUniversityFundingModel(MTCU,2015)emphasizesanincreasedemphasisontyingfundingtoevidenceoflearningquality,specificallyidentifyingtheassessmentoflearningoutcomesasakeystrategicdirectionforchangetotheuniversityfundingmodelintheprovince,noting:

Understandingwhatstudentsknow–andwhattheyshouldknow–asaresultoftheirtimeatuniversityiscriticaltoaddressingquality.Measuringandassessingundergraduatelearningoutcomeshasthepotentialtoaddconsiderablevaluetothesector,enablingstudentstounderstandwhattheyhavelearned,governmentstounderstandwhatskillsarebeinggenerated,anduniversitiestodrive


4

continuousimprovement.ItisforthesereasonsthatpreviousrecommendationstotheOntariogovernmenthaveidentifiedsuchassessmentasanimportanttodeterminingthevalueaddedthrougheducation(p.44).

Accordingly,theFundingModeldocumentadvocatedfortheaccelerationofcurrentsectoralworkonlearningoutcomes,andfortheprioritizationoflearningoutcomesassessmentasaconditionoffunding.

Learningoutcomesarealsoviewedprovinciallyasakeyleverforimprovingstudentmobilityandsharedcourseandprogramdevelopmentthroughcredittransfer(Fallon,2015),whichrequiresmechanismsforaccuratecomparisonofcurriculumcontentatthecourse,year,andprogramlevels.In2011,themember-drivenOntarioCouncilonArticulationandTransfer(ONCAT)wasestablished,withamissiontoenhanceacademicpathwaysandreducebarriersforstudentslookingtotransferamongcollegesoruniversities.ONCAThasestablishedaLearningOutcomesSteeringCommitteewhichhasfundednumerouslearning-outcomesfocusedpathwayprojects,andsponsoredlearning-outcomesrelatedcolloquia.ONCATseesitsroleasassistingintheexchangeofinformationandcommunicationinordertoenhancemutualunderstandingofeachother’sstudentsandprograms:

Learningoutcomesfacilitateaclearcomparisonofcredentialsatthecourse,year,andprogramlevelsand,assuch,canplayavitalroleincredittransfer,bybothimprovingexistingpathwaystomaximizestudentsuccessandbyincreasingtheoverallnumberoftransferopportunities.Whendisciplineexpertsfromvariousinstitutions,sectors,andjurisdictionscometogethertodiscusstheirsubjectareathroughalearningoutcomeslens,theygaingreaterunderstandingofwhatisexpectedofstudentsineachprogram.Theclarityandtrustthatresultsfromsuchworkenablespartnerstomoreconfidentlybuildpartnershipsandpathwaysamongprogramsandinstitutions,whichultimatelyexpeditestheoftenlengthyandcomplexarticulationprocess(Fallon(2015,p.2)

Asinmanyjurisdictions,thedevelopmentoflearningoutcomesismoreadvancedthanthedevelopmentofsystematicandalignedassessmentoflearningoutcomes.Althoughnotprimafacievalidorconsistentwithscholarlyrecommendationsregardingassessment,throughHEQCO,universitiesandcollegeshavebeeninvolvedinanumberofpilotsofstandardizedtestsintendedtomeasurestudentachievementofgraduateattributessuchaswritingskillsandcriticalthinking,includingtheAHELOstudydescribedabove,apilotoftheCollegiateLearningAssessment,andacurrentinitiativeinvolvingtheOECDPIACCSurveyofAdultskills.Anumberofprovincialprojectsareexploringthechallengeofoutcomesassessmentatvariousmoregranularandmulti-facetedlevels.Theuseofe-portfolios,capstonecoursesandtheOdetteAssuranceofLearninginitiativeareexamplesofother,moremulti-facetedandintegratedapproaches.

In2012,HEQCOestablishedasix-institutionconsortiumtoexploretheassessmentoflearningoutcomes,whichincludesDurham,GeorgeBrownandHumberColleges,andQueen’s,theUniversitiesofTorontoandGuelph.Theinstitutionsinvolvedareeachassessingdifferentbutsomewhatoverlappingskillssets,includingcommunication,criticalthinking,literacy,teamwork,problemsolving,professionalandethicalbehaviour,lifelonglearning,design,andinvestigation.Consortiummembershavedevelopedandtestedavarietyoftoolsincludinge-portfolios,studentself-assessment,rubrics,standardizedmeasuresandtests,web-applicationsforoutcomestracking,andlearninganalyticsdata.RelatedprojectdescriptionscanbefoundinAppendixC.AnothermajorinitiativesawthepublicationofLearningOutcomesAssessment:APractitioner’sHandbook(Goffetal.,2015),acollaborationinvolving


5

numerousuniversitiesandcolleges,withsignificantUniversityofWindsorleadershipthroughthecontributionsofMichaelK.Potter,ErikaKustra,andJessicaRaffoul.


6

AppendixC:FundedLearningOutcomesInitiativesinOntarioHigherEducationQualityCouncilofOntarioLearningOutcomesConsortiumInitiatives

LearningOutcomesAssessmentandProgramImprovementatQueen'sUniversityPrincipalInvestigator(s):Queen’sUniversity(LeadResearcher(s):JillScott,BrianFrankandNatalieSimper)Thisstudywillconductalongitudinalassessmentofgenerallearningoutcomessuchascriticalthinking,problemsolvingandcommunication.Theresearchwillusefiveinstrumentstomeasurespecificaspectsofthelearningoutcomesandcognitiveskills.Qualitativeandquantitativedataanalysiswillbeusedtoprovideinstructorswithfeedbackontheirstudents’demonstrationofthelearningoutcomes.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthatarescalableattheinstitutionallevel.Value:$1,103,999.00APilotProjectfortheDevelopmentofanOnlineLearningOutcomeAssessmentStrategyfortheUniversityofGuelphPrincipalInvestigator(s):UniversityofGuelph(LeadResearcher(s):SergeDesmarais)Thisprojectwilldevelopanonlinelearningoutcomescapturingandassessmenttool;pilot-testitssuitability;andconductaninitialstudyusingthetooltomeasurestudents’skillacquisitionovertime.Theonlineassessmentstrategywilleventuallybeimplementedacrosstheentireinstitutionandisexpectedtobepartoftheregularqualityassuranceprocess.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthatarescalableattheinstitutionallevel.Value:$1,550,800.00CapacitytoMeasureEssentialEmployabilitySkillsPrincipalInvestigator(s):GeorgeBrownCollege(LeadResearcher(s):GaryKapelus)Thisprojectdevelops,pilotsandteststools,learningactivitiesandassessmentstrategiestomeasurestudentachievementofessentialemployabilityskillsrelatedtocriticalthinkingandcommunication.Theintentistocreateapracticalresourcefortheassessmentofcriticalthinkingandcommunicationthatcanbeappliedinanycurriculumacrossthecollegeand,ultimately,anypostsecondarysetting.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthatarescalableattheinstitutionallevel.Value:$240,525.00DevelopRubrictoAssessSkillsinaCollege-WideApplicationPrincipalInvestigator(s):HumberCollege(LeadResearcher(s):PatriciaMorgan)Thisstudyseekstodevelopandvalidateaneasy-to-usescorecardtomeasurestudentoutcomesincriticalthinkingandcommunicationskills.Thescorecardwillbepilotedintwoprograms(policefoundationsandbusinessadministration)beforebeingimplementedinalldiplomaanddegreeprogramsasaregulardiagnostictooltotrackstudents’progressterm-over-term.Thisfive-phaseprojectinvolvesanassessmentofotherrubricsandscorecards,thedevelopmentoftheassessmenttool,anevaluationofthetoolandapossibleexpansionofthescorecardinallofHumber’sschoolsandprograms.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthat


7

arescalableattheinstitutionallevel.Value:$293,906.00DevelopmentofAnalyticRubricsforLearningOutcomesAssessmentPrincipalInvestigator(s):UniversityofToronto(LeadResearcher(s):SusanMcCahan)Thisstudyseekstodeveloparesourcetoassessstudentcompetenciesincommunication,teamwork,investigation,designandproblemanalysis.Thisresourcewillbedesignedinawaythatwillmakeitapplicableinawiderangeofcoursesandinstitutions.Theassessmenttoolwillbepilotedinnineengineeringprogramsandstudentfocusgroupswillbeconductedtoassesswhetherthestudentsunderstandthefeedbackprovidedbytherubricsinthewayintendedbytheinstructor.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthatarescalableattheinstitutionallevel.Value:$752,022.00DurhamCollegeStudentSuccessePortfolio:EssentialEmployabilitySkillsProjectPrincipalInvestigator(s):DurhamCollege(LeadResearcher(s):JacquelineTowellandChrisHinton)Thisstudyexploresstudent,industryandfacultyexperiencesusingastudentsuccesseportfolio.ThisresearchseekstounderstandtheextenttowhichtheeportfolioiseffectiveinassessingthedevelopmentoftheMinistryofTraining,CollegesandUniversities’EssentialEmployabilitySkills,asetofgenericlearningskillssuchascommunication,criticalthinkingandteamwork.Thisthree-phaseprojectinvolvesresearchingdifferenteportfoliooptions,pilotinganeportfolioplatformandgainingfacultyandstudentfeedbackthroughsurveysandfocusgroups.ThisstudyispartofHEQCO’sLearningOutcomesAssessmentConsortiumcomposedofsixpostsecondaryinstitutionsintheprovince.Theirgoalistopilotassessmenttoolsandtechniquesthatarescalableattheinstitutionallevel.Value:$101,100.00HigherEducationQualityCouncilofOntarioPublicationsrelatedtoLearningOutcomesAvailableat:http://www.heqco.ca/en-ca/Research/ResPub/Pages/default.aspx#Default={%22k%22%3A%22%22%2C%22r%22%3A[{%22n%22%3A%22PublicationSubject%22%2C%22t%22%3A[%22\%22%C7%82%C7%824c6561726e696e67204f7574636f6d6573\%22%22]%2C%22o%22%3A%22and%22%2C%22k%22%3Afalse%2C%22m%22%3Anull}]}TheLanguageofLearningOutcomes:DefinitionsandAssessmentsSarahBrumwell,FionaDellerandAlexandraMacFarlane,HigherEducationQualityCouncilofOntarioOutcomesBasedEducationInitiativesinOntarioPostsecondaryEducationCaseStudiesQinLiu,UniversityofTorontoOutcomesbasedfunding:Currentstatus,promisingpractices,andemergingtrendsMaryZiskin,UniversityofDayton,DonHosslerandKarynRabourn,IndianaUniversity,OsmanCekic,CanakkaleOnsekizMartUniversityandYoungsikHwang,IndianaUniversityEngagingStudentstoThinkCriticallyinaLargeHistoryClassMairiCowan,TylerEvans-Tokaryk,ElaineGoettler,JeffreyGraham,ChristopherLandon,SimoneLaughton,SharonMarjadsingh,CaspianSawczakandAlisonWeir,UniversityofTorontoMississauga


8

@IssuePaperNo.19-EmphasizingNumeracyasanEssentialSkillNicholasDion,HigherEducationQualityCouncilofOntarioEvaluatingEssentialSkillsforOntario'sTradespeople(ESOT)ProjectBeaClarkandMartiJurmain,CollegeSectorCommitteeforAdultUpgradingUndergraduatesUnderstandingofSkillbasedLearningOutcomes:Cane-portfoliosHelp?TanyaS.MartiniandMattClare,BrockUniversityTuning:IdentifyingandMeasuringSector-BasedLearningOutcomesinPostsecondaryEducationMaryCatharineLennon,BrianFrank,JamesHumphreys,RhondaLenton,KirstenMadsen,AbdelwahabOmriandRoderickTurnerProductivityImplicationsofaShifttoCompetency-BasedEducation:AnenvironmentalscanandreviewoftherelevantliteratureBrianAbner,OksanaBartoshandCharlesUngerleider,DirectionsEvidenceandPolicyResearchGroup,LLP,withtheassistanceofRobTiffinEvaluatingCriticalThinkingandProblemSolvinginLargeClasses:ModelElicitingActivitiesforCriticalThinkingDevelopmentJamesKaupp,BrianFrankandAnnChen,Queen’sUniversityAHELO:TheOntarioExperienceMaryCatharineLennonandLindaJonker,HigherEducationQualityCouncilofOntario(HEQCO)•PilotingtheCLAinOntarioMaryCatharineLennon,HigherEducationQualityCouncilofOntario(HEQCO)TakingLearningOutcomestotheGym:AnAssignment-BasedApproachtoDevelopingandAssessingLearningOutcomesSteveJoordens,DwaynePareandLisa-MarieCollimore,AdvancedLearningTechnologiesLab,UniversityofTorontoInformationLiteracyCompetencyStandardsforStudents:AMeasureoftheEffectivenessofInformationLiteracyInitiativesinHigherEducationAmandaDuncanandJenniferVarcoefromGeorgianCollegeCollege-LevelLiteracy:AnInventoryofCurrentPracticesatOntario'sCollegesRogerFisherandWhitneyHoth,FanshaweCollege@IssuePaperNo2-2AFineBalanceMaryCatharineLennon@IssuePaperNo2-3SignallingAbilitiesandAchievementMaryCatharineLennonHEQCOhasalsotakenanactiveroleintheorganizationanddevelopmentofbotheventsandresources


9

intendedtosupportthoseseekingtoadoptoutcomes-basedapproachesthrough.Thesehaveincludedanumberofsymposiaaswellasonlineanddownloadableresourcessuchasthefollowing:Measuringmatters:AssessinglearningoutcomesinhighereducationWebinarserieshttp://www.heqco.ca/en-ca/OurPriorities/LearningOutcomes/Pages/Measuring-matters-Assessing-learning-outcomes-in-higher-education.aspxLearningOutcomesAssessment:APractitioner’sHandbookMarch10,2015LoriGoff,MichaelK.Potter,EleanorPierre,ThomasCarey,AmyGullage,ErikaKustra,RebeccaLee,ValerieLopes,LeslieMarshall,LynnMartin,JessicaRaffoul,AbeerSiddiqui,GregVanGastelOntarioCouncilonArticulationandTransferprojectscanberetrievedfrom:http://www.oncat.ca/index_en.php?page=research


10

AppendixD:Practices,policies,andproceduresinacademicunits

Procedures Supports&Resources Policies&Documents AssessmentofLO’s EffectivePracticesIdentifiedEDUCATION AssociateDeansdirectlyresponsiblefor

processFornewcoursesandprograms,LOsaredevelopedbyrelevantfacultymembers/committeesandreviewedbyrelevantAssociateDean,inpreparationoftherelevantPDCform.OncePDCformiscomplete,LOsareapprovedbytheFacultyCouncilaspartofnormalprocess.Forexistingcourses,LOsaredevelopedbyinstructorsresponsibleaspartofthecoursesyllabusdesign,andreviewedbytherelevantAssociateDean.

Forpre-servicecourses,learningoutcomesaredevelopedinalignmentwiththeOntarioCurriculumguidelines

Developingrevisedsyllabustemplate

Assessmentsarealignedtooutcomes;thisisafoundationalprincipleofteachingandourstudentslearnthisaspartoftheirownteachingpracticeHadanexitsurveyrecently,AssociateDeansgatherandreviewallcourseoutlines.SETscoresanddetailsprovidesomefeedback

CommondevelopmentofcoursesyllabustemplatewithalignmentofLOsandassessmentKnowledgeofcurriculummapping-teachconcept

ENGINEERING ProcesssupportedbyFacultyofEngineeringCurriculumCommitteeandUndergraduateProgramsCoordinator.TheCanadianEngineeringAccreditationBoard(CEAB)requiresthatwetracktheachievementof12GraduateAttributes(GrAtts)necessaryforstudentsgraduatingfromanaccredited,4-year,engineeringprogram.ItissuggestedthattheGrAttsbesub-dividedintoapproximatelythreeindicatorseach.Course-levellearningoutcomesarethenmappedtotheindicators.UndergraduateProgramsCoordinator(UPC)hasmetwithindividualinstructorstoreviewthealignmentoftheircourselearningoutcomestotheCEABGrAttsandindicators.NewLOsthensubmittedtoregularSenateprocess

CommonSyllabustemplateincludingLOsAdoptedacommonsetofindicatorswhichhadbeenpostedbytheUniversityofTorontoontheEGADwebsite,http://egad.engineering.queensu.ca/?page_id=1207.Havecentralspacetostorefileselectronically

ACEABcourseinformationsheetiscompletedforeachcoursetoreporttheGrAttsandindicatorsforeachcourse.Theyalsoindicatethepercentageoftimethatisdevotedtospecificrequiredareas.

CEABaccreditationrequiresbothinput-based(i.e.,NS,Math,etc.)andoutput-based(i.e.,GrAtts)reporting.

Everyinstructorisrequiredtomaintainabinderofinformationforeachcoursethathe/sheteaches.Thisbindercontainsthecoursesyllabus;lecturenotes;samplesofstudentworkforallassignments,tests,quizzes,reports,projects;marking;anddataforallcourseLOs.

Eachprogrammapswheretheindicators(andassociatedGrAtts)arebeingassessed.Specificassessmentsarereported,alongwiththenumberofstudentswhoexceedexpectations(EE),meetexpectations(ME),anddonotmeetexpectations(DNM).Mostcoursesusepercentagecut-offsfordeterminingtheselevels.CurriculummapsidentifywhereLOsareassessedAssessmentsofGrAttindicatorsarecollectedusingExcelspreadsheets.Histogramsoftheassessmentresultsforeachcoursearegeneratedmanuallyanddistributedtotheprograms.

Everycourseusesacommonsyllabustemplatethatincludesalistofthecourse-levellearningoutcomesandhowtheyrelatetoourCEABGrAttsandindicators.ThetrackingofCEABGrAttsandindicatorsisledbytheFacultyofEngineeringCurriculumCommittee(FECC).ThisallowsforthesharingofbestpracticesamongsttheprogramsandforconsistentreportingthroughouttheFaculty.AnUndergraduateProgramsCoordinatorprovidessupportandisresponsiblefortheprocessGenerativeCurriculummapsaidsinidentifyingwhereattributes/outcomesarebeingassessed


11

Procedures Supports&Resources Policies&Documents AssessmentofLO’s EffectivePracticesIdentifiedFAHSS Processandpersonresponsiblevaryby

Department.Acommonprocess:Consultwithcolleagues,CTL,curriculumorundergraduatecommittee/graduatecommittee,andthenreviewLOandapproveatFacultyCouncilbeforeregularprocessatPDC

Othercoursesasmodels,colleagues,CTL,Undergraduate/graduatecommittee,DepartmentalCouncil,FCC/Gradcouncil

Coursesyllabimustalignwithprogramoutcomes.Outcomesmustbepostedonthecoursesyllabus.(Note:Mostsurveyeddidnotidentifyanypoliciesordocuments)DocumentcontainingLOsfor6differentprograms

Generally,thereisnospecificongoingassessmentprocessinplacebesidesthenormalexamsandassessmentsinthecourses.Thisinformationisnotaggregatedoranalyzed.Program-levellearningoutcomesarenotassessedseparately.Someprogramsusealumnisurveys.Someprogramshavebeguninitialmapping.

TheFacultyhasadoptedacoursesyllabustemplatethatincludesLOsFourdepartmentsindicatedtheydevelopedcurriculummapswhichidentifythevariousmethodsofassessmentusedacrosscourseofferings

HK DepartmentHeadhasresponsibilityforoversightForanewcourse,afacultymemberwouldputtheappropriatePDCformtogethercompletewithLOs.Oncecomplete,itwouldtotheDepartmentHeadforreview,thenCTLforreviewoftheLOs,thenPDCNosystematicprocessfornewprograms

CTLpersonnelPreviousexamplesofsyllabiSomemayalsousematerialsfromthedepartment-specificLOworkshopfromafewyearsago

Nostandardpolicies.Requestthatsomestandarditemsareincludedineverycoursesyllabus(e.g.,relevantSenatebylaws).CoursesyllabiforeachtermarecollectedcentrallyinKinesiologyandstored.Thishelpswhenstudentswhograduatecontactustotrackdownasyllabusfromacoursetheytookpreviouslywhenapplyingtoanewprogram.

ManyinstructorslinkLOswithassessments,butthisisNotdonesystematicallyacrossallcourses.Currently,program-levelLOsarenotreallybeingassessed.However,thisisintheplanmovingforward.

CurrentlyevaluatingLOsandprocessesthroughaCLIFgrantfromtheCTL,toassessallcoursesyllabifromthelast5years.OneofthepartsoftheCLIFgrantanalysisistolookatcomplianceregardingthestandarditems.OnceCLIFgrantanalysisiscomplete,HKplanstoholdaworkshopforeveryoneinKinesiologytogoovertheresultsandprovideaplanforadvancingpractices.Theintentoftheapproachisnottodevelopasyllabustemplate,butrathertoassess(inadditiontoLOs)severalthingsrelatedtotheHKcurriculum.Forexample,dowehavegapsthatneedtobefilled?Whattypeofassessmentsarebeingused?IsthereaprogressioninLOsasthestudents’progressfromfirsttofourthyearcourses?TheplanistocontinuetolookatmappingLOsthroughouttheprogramsandalignLOswithassessmentsinallclasses.


12

Procedures Supports&Resources Policies&Documents AssessmentofLO’s EffectivePracticesIdentifiedLAW Eachfive-yearstrategicplanbeginswith

developmentofprogram-levellearningoutcomes.Thereisalearningoutcomessubcommittee.i.Academicplanningmeeting(monthlymeeting)ii.Facultycouncil(monthlymeeting)iii.FormsgotoPDC

CTLaskedtoreviewLOsearlyintheprocess

Nopoliciesidentified. Noformalassessmentgathering,butinitialmappingofassessmentmethodshastakenplace.

EngagedasaFacultyretreatsandinareviewofexistingsyllabitomapthecurriculumandtoidentifygapsandstrengths.Alsomappedassessmentmethods.InitiatedbyfacultymembersandfacilitatedbyCTLmember.

NURSING AssociateDeanhasresponsibilityforoversight.LOsnotrequiredforeachcourse,thoughthereareyearleveloutcomesforaccreditation.Competenciesaredevelopedforclinicalcourses.IndividualfacultydevelopLOsifdesired,submittoCurriculumCommittee.

TheOntarioCollegeofNursescompetenciesforexternalprofessionalaccreditationBloomsTaxonomytoassistinconceptualizinglearningoutcomesatdifferentlevels.

OntarioCollegesofNursingaccreditationstandardsandthe120competenciesthatgraduatingnursesneedtomeet.StandardsyllabususedbythecollaborativeprogramwithLambtonandStClair,butthisdoesnotapplytoWindsor-onlycoursesthatarenotpartoftheprogram.

Notgenerallyevaluated.Inevaluationofclinicalcompetencies,however,thecompetenciesmustbedirectlyobservedinaclinicalsetting,sothereissomealignment,evenifitisnotexplicitlyrecognized.NCLEXexamgivessomeindicationofprogram-leveloutcomes,andtheyareabletogetdatabackonhowtheirstudentsdidontheexamsocurricularadjustmentscouldbemade.

Alargevarietyofassessmentmethodsused,includingmethodsauthentictothediscipline(OSCIetc.)Startedcurriculummapprocess(thoughdelayedforaccreditation)


13

Procedures Supports&Resources Policies&Documents AssessmentofLO’s EffectivePracticesIdentifiedSCIENCE Program-levelLOstendtobedeveloped

byoneortwoindividuals(Head,UndergraduateChair,facultymember).Developmentofcourse-levelLOstendstobetheresponsibilityoftheindividual(s)whoteachesaparticularcourse.TheLOsaresubmittedforreviewtotheundergraduatechair/committeeandCTLbeforeapprovalthroughtheDepartmentCouncil,ScienceProgramDevelopmentCommittee(SPDC),ScienceFacultyCouncil,andthenregularSenatechannels

CTLandScienceProgramDevelopmentCommittee(SPDC)inanadvisorycapacityforfeedback

Discussingthedevelopmentofastandardsyllabus

Program-levellearningoutcomesarenotpresentlybeingassessed.Inmanycases,theyareintheprocessofbeingdeveloped.Itisexpectedthatthefocuswillchangetoamorein-depthstudyofthelearningoutcomesintermsofhowtheyareassessedonceallprogramshavelearningoutcomes

Thedevelopmentofacurriculummapforeachundergraduateprograminitiated10yearsago-interesttocontinuedevelopingthistool,incorporatingLOinformationtoclearlyidentifythefulfillmentofallgraduateattributesbytheendofaprogram.Faculty-ledretreatstodetermineLOstogether.


14

AppendixE:UniversityofWindsorQualityAssuranceFlowchart

!!!!

!

1!includes!opportunity!for!revisions!and!response!to!external!review!to!be!forwarded!to!Departmental!Council!2!the!department!submits!biennial!updates!on!progress!on!recommendations!for!UPR!Subcommittee!and!PDC!review!***outcomes!of!the!Quality!Council!reviews!will!be!reported!to!Senate***!

Adapted!from%the%University%of%Ontario%Institute%of%Technology%Quality%Assurance%Handbook,%2010

UPR!Subcommittee!of!PDC2!

Departmental!Council(s)!(or!Faculty!Council(s),!for!nonGdepartmentalized!faculties)!

Faculty!Coordinating!Council(s)!(for!departmentalized!faculties)!

Program!Development!Committee!

Senate!

External!Review1!

Graduate!Studies!Council!(for!graduate!programs!and!courses)!

Minor!course!calendar!change!

PDC*Form*E*

New!nonGdegree!undergraduate!diploma!or!certificate!programs!

PDC*Form*A*

Minor!program!change!and/or!New!Course!Proposal!

PDC*Form*C*and/or**PDC**Form*D*

!

New!forGcredit!graduate!diploma,!graduate!collaborative!program,!&!endorsement!of!new!field!in!existing!graduate!programs!

PDC*Form*A*

Major!modification!to!existing!program!

PDC*Form*B*

New!degree!program!

PDC*Form*A*

Program!review!

Provost!

Senate!(summary!&!recommendations!for!

information)!Quality!Council!

Quality!Council!(summary!&!recommendations)!

Quality!Council!(expedited!process)![for%graduate%diplomas,%

%graduate%collaborative%programs,%and%graduate%fields%only]%

PDC!for!information!

only!

MTCU!for!funding!approval,!as!required!

External!Review!!

1. Response!from!AAUHead(s)!to!original!review

2. Departmental!Council(s)(as!required!by!area(s))

1. Response!from!Dean(s)to!original!review

2. Faculty!CoordinatingCouncil(s)!(as!required!byFaculty(ies))

SelfGstudy!

Response!from!:!1. AAU(s)!(proposers)2. Dean(s)

Graduate!Studies!Council!(for!

graduate!courses)!

Provost!

MTCU!for!funding!approval,!as!required!

!