Learner Centered Assessment in Online Course in Online Courses ... Design anddeliverinstructionthatisresponsivetostudent ... learner-centered assessment:

  • Published on
    15-Mar-2018

  • View
    217

  • Download
    5

Transcript

<ul><li><p>10/29/2014</p><p>1</p><p>LearnerCenteredAssessmentinOnline</p><p>CoursesUsingSelfEvaluationandPeerReviewtoDocumentStudentLearningOutcomes</p><p>Shawntel Landry, Ed. D.Alana Sloan, Ed. D.</p><p>Assessment InstituteOctober 21, 2014</p><p>Agenda1. Background/Set the Stage2. Assessment System3. Learner-Centered Assessment4. Learner-Centered Assessment: Study 15. Learner-Centered Assessment: Study 26. Overall Findings and Continuous Improvement</p></li><li><p>10/29/2014</p><p>2</p><p>BackgroundandSettheStage</p><p>AmericanCollegeofEducation</p><p> Regionally accredited graduate-level education college</p><p> Located in Indianapolis, IN</p><p> Programso 16 graduate certificateso 23 masters programso 1 specialist programo 1 doctoral program </p></li><li><p>10/29/2014</p><p>3</p><p>SettingtheStage Moved in 2013 from a proprietary learning management </p><p>system (LMS) to Canvas LMS Increased interactivity and opportunity for learner-centered </p><p>assessment through more automated survey capabilities and peer review assessment</p><p> Prior to 2013:- pre- and post- self-assessments were in Word documents, and data were analyzed by hand. </p><p>- peer reviews could only be conducted in discussion boards.</p><p>GuidingQuestions How can learner-centered assessment provide </p><p>evidence of student mastery of learning outcomes?</p><p> What measures can serve as rich data sources?</p></li><li><p>10/29/2014</p><p>4</p><p>ResearchQuestionCan learner-centered assessment be used to provide evidence of student mastery of learning outcomes? </p><p>Lets take a poll</p><p>PollResults</p></li><li><p>10/29/2014</p><p>5</p><p>AssessmentSystem</p><p>AssessmentSystemFourComponents</p><p>StudentLearningOutcomes</p><p>Quantitative</p><p>Qualitative</p><p>Performance</p><p>LearnerCentered</p></li><li><p>10/29/2014</p><p>6</p><p>SLOAlignment</p><p>AcademicOutcomes</p><p>ProgramOutcomes</p><p>CourseObjectives</p><p>ModuleObjectives</p><p>rubricssurveysinstitutionwide</p><p>stateandnat.standardsdisciplinespecificstandards</p><p>ProgramBlueprintandSLOAlignments</p></li><li><p>10/29/2014</p><p>7</p><p>AssessmentAlignment Course Level</p><p>o Course TOTS Alignment Map</p><p>o Course Alignment Map</p><p> Program Levelo Program Alignment Map</p><p> Course Levelo Course Assessment Alignment </p><p>Map (Signature Assessments, Peer Review)</p><p>AmericanCollegeofEducationT2T/MATProgramAlignmentMap</p><p>Revised2.14.12</p><p>ProgramOutcomesCourseObjectives</p><p>EDTT520 EDTT504 EDTT506 EDTT507EDTT508</p><p>EDTT511</p><p>EDTT502</p><p>DL501</p><p>EDTT513 EDTT516 EDTT501 EDTT515</p><p>EDTT505</p><p>%ofEmphasis</p><p>1.Designanddeliverinstructionthatisresponsivetostudentdiversityanddifferencesandpromoteshighachievementforallstudents.</p><p>15,16 1,2,3,4,5,6,7,8,9,11,12,13,14,16,</p><p>17</p><p>1,2,3,6,7,8,9,11,12,13,14,15,16,17,18</p><p>3,7,12</p><p>4,5,6,8,11</p><p>1,9,12,13,15,16</p><p>2,4,6,10</p><p> 3,4,6,7</p><p>1,4,6,7,8,9,10,15</p><p>5,6 </p><p>23%</p><p>2.Usestandardsbased,datadrivendecisionmakingandintegrated,differentiatedinstructionthateffectivelyintegratesappropriatetechnologiesandtoolsandsupportsthelearningofallstudents.</p><p>4,6,7,18,19,20,23</p><p>7,11 12,18 2,4,8,9,10,13</p><p>4,5,6,8,11</p><p>2,3,4,6,7,8,17</p><p>1,4,6,10</p><p>1,2,3,4,5</p><p>3,4 4,6,7,8,9,10,14,17,18,19,20,21</p><p>3 2</p><p>19%</p><p>3.Explainthemultipleformsofassessment,anduseassessmentresultstoinforminstructionaldecisionmaking.</p><p>2,13,14,21,22</p><p>4 4,5 5,6,10,12,14</p><p>3 8,10 11 7 1,2,7 22,23 1,2,3,4,5,6,7</p><p>1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22</p><p>4</p><p>19%</p><p>4.Constructwellorganized,positivelymanaged,andsafelearningenvironmentsthatpositivelyimpactinstructionaleffectivenessandstudentachievement.</p><p> 2,10,15 11 7 18 7,8 2,3,4,5</p><p>5 2,3,11,12,13,14</p><p> 1,3,58%</p><p>5.Validateparticipationinactivitiesthatpromotecontinuousprofessionalgrowth,inquiry,research,andcollaborationwithotherstoimprovestudentachievementandtomaintainthehighethicalstandardsoftheteachingprofession.</p><p>3,8,9,24,25,26</p><p>10,18 10 3,14 14 1,3,5,9,12,13,14</p><p>6 1,2 2,3,5,16,25,26,27,28,29,30,31,32,33,</p><p>34</p><p> 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22</p><p>4</p><p>21%</p><p>6.Applyknowledgeofscientificallybasedresearchtoplananddeliverinstructionincontentareas,includingreading,languagearts,mathematics,science,socialstudies,finearts,health,wellness,andphysicaleducation,forthepurposeofimprovingtheachievementofallstudents.</p><p>1,5,10,11,12,17</p><p>1,2,3,5,6,9,12,14,16,17</p><p> 1 1,2,9,10,11</p><p>4,11 8 6 </p><p>9%</p><p>LearnerCentered</p><p>Assessment</p></li><li><p>10/29/2014</p><p>8</p><p>LearnerCenteredAssessment</p><p>An activity designed by the professor that yields comprehensive information for analyzing, discussing, and judging a learners performance on valued abilities and skills (p. 12)</p><p>- Huba &amp; Freed (2000)</p><p>Lets take a poll</p><p>PollResults</p></li><li><p>10/29/2014</p><p>9</p><p>BenefitsofaLearnerCenteredApproach</p><p>At the student level: Students become active participants rather than </p><p>passive consumers of content (p. 903).- Bayat (2012)</p><p> Learner-centered assessment:- helps students synthesize learning experiences.</p><p>- Huba &amp; Freed (2000)</p><p>BenefitsofaLearnerCenteredApproach</p><p>At the institutional level, learner-centered assessment:- promotes high expectations in the learning environment.</p><p>- demonstrates respect for diverse talents and learning styles.</p><p>- promotes coherence in providing data to improvecurriculum and guide revisions.</p><p>- Huba &amp; Freed (2000)</p></li><li><p>10/29/2014</p><p>10</p><p>LearnerCenteredAssessment:TwoStudies</p><p>1. Pre- and Post-Course Self-Evaluation to measure Student Learning Outcomes (SLOs)</p><p>2. Peer Review Assessment to measure SLOs</p><p>LearnerCenteredAssessment:Study1</p><p>(SelfAssessment)</p></li><li><p>10/29/2014</p><p>11</p><p>SelfAssessmentDefinitionsThe involvement of learners in making judgments about their own learning, particularly about their achievements and the outcomes of their learning </p><p>- Boud &amp; Falchikov (1989)</p><p>Evaluations learners make about their current knowledge levels or increases in their knowledge levels in a particular domain.</p><p>- Sitzman, Ely, Brown, &amp; Bauer (2010)</p><p>BenefitsofSelfAssessment</p><p>Debate: Are self-assessment results more representative of cognitive or affective learning growth?</p><p>Self-assessment can be used as a triangulation technique to supplement information (about student learning) acquired through direct measures.</p><p>- Moskal (2010)</p><p>Change in students psychological perspectives resulting in: Metacognitive knowledge Learning constructivism Development of self-efficacy</p><p>- Basnet, Basson, Hobohm, &amp; Cochrane (2012)</p></li><li><p>10/29/2014</p><p>12</p><p>BenefitsofSelfAssessment</p><p>Self-assessments of knowledge have a critical role in the learning process; learners benefit from having an accurate understanding of their knowledge levels.</p><p>- Dunning et al. as cited in Sitzman et al. (2010)</p><p>GuidingQuestions What data can we harvest from pre- and post-</p><p>course self-evaluations? How can we graphically show this data? </p></li><li><p>10/29/2014</p><p>13</p><p>ResearchQuestion Can self-assessment provide evidence of mastery of </p><p>student learning outcomes? </p><p>Study1Design Three programs M. Ed. in Curriculum and </p><p>Instruction, Educational Leadership, Educational Technology </p><p> Random term selected from 8 terms in 2013 From the term, 10% of the total number of courses </p><p>randomly selected from a numbered course list Pre- and Post- Self-Evaluations collected and </p><p>analyzed from all student sections of selected courses</p></li><li><p>10/29/2014</p><p>14</p><p>Study1DesignCurriculum&amp;Instruction</p><p>EducationalLeadership EducationalTechnology</p><p>Year 6.3.2013 11.18.2013 4.22.2013Term(18) 4 8 3NumberofCoursesPerTermPerProgram</p><p>8 5 3</p><p>NumbersandTitlesofCoursesPerTerm</p><p>1.CI50332.CI53133.CI53334.CI53535.CI53936.CI54237.CI54538.CI5503</p><p>1.EL50332.EL56233.EL56534.EL57535.EL5903</p><p>1.ET50132.ET50233.ET5033</p><p>RandomlySelectedCourseNumbers(10%oftotalcoursesperselectedterm)</p><p>5.CI5393 2.EL5623 3.ET5033</p><p>NumberofSectionsPerSelectedCourse</p><p>2(56) 5(153) 1(30)</p><p>NumberofStudentsCompletingPre andPostSelfEvaluations</p><p>Pre 35Post 29</p><p>Pre 118Post 101</p><p>Pre 22Post 21</p><p>DataComparisons:C&amp;I</p></li><li><p>10/29/2014</p><p>15</p><p>DataComparisons:EL</p><p>DataComparisons:ET</p></li><li><p>10/29/2014</p><p>16</p><p>Findings In all selected courses and across programs, students self-</p><p>reported significant growth in the acquisition of student learning outcomes when pre- and post- results were compared.</p><p> Across programs, post-course results showed an overwhelming majority of students rating their knowledge level as a 3 or 4.</p><p>4-Point Likert Scale1. I am unfamiliar with this concept.2. I have heard of this but need to learn more.3. I am fairly confident in my knowledge of this and can </p><p>implement this on my own.4. I can teach or consult with someone else to help them </p><p>implement this.</p><p>Findings:Item1PerceivedGrowthComparison</p><p>0 20 40 60 80 100 120</p><p>Curriculum&amp;Instruction</p><p>EducationalLeadership</p><p>EducationalTechnology</p><p>PostPre</p></li><li><p>10/29/2014</p><p>17</p><p>LearnerCenteredAssessment:Study2</p><p>(PeerAssessment)</p><p>PeerAssessmentA method in which students evaluate each others work based on set criteria</p><p>- Falchikov (2001) &amp; Gouli et al. (2006)</p></li><li><p>10/29/2014</p><p>18</p><p>BenefitsofPeerAssessment</p><p>Peer evaluations help students who are being evaluated by adding insight of people (fellow students) with different perspectives than their professor.</p><p>The act of evaluating peer work can strengthen critical thinking skills and help students understand faculty expectations</p><p>- Suskie &amp; Banta (2009)</p><p>GuidingQuestions What is the level of student involvement in the peer </p><p>review assessment process? To what extent are students applying peer </p><p>feedback to improve their work prior to final submission?</p></li><li><p>10/29/2014</p><p>19</p><p>ResearchQuestion How can peer assessment provide evidence of </p><p>student mastery of learning outcomes?</p><p>PeerReviewAssessmentSettingtheStage</p><p> Signature assessments are chosen by department chairs for inclusion in selected courses. - Are assignments appropriate for inclusion in the students portfolio?</p><p> All signature assessments are aligned to course objectives and learning outcomes. </p></li><li><p>10/29/2014</p><p>20</p><p>PeerReviewAssessmentSettingtheStage</p><p> Peer review is an interactive capability within the Canvas LMS.</p><p> Currently, 38 courses in masters programs have signature assessments with a peer review component. </p><p> Peer review is a pre-step to final submission of a signature assessment. The peer review portion of the assignment is non-graded.</p><p> The professor assigns peer review partners in the LMS. Each student is directed to submit the draft of the </p><p>signature assessment for peer review. Partners review and critique each others papers.</p><p>PeerReviewAssessmentSettingtheStage</p><p> Peer review tools:- Rubric aligned to academic outcomes (identical to the rubric the professor uses to grade final signature assessment)</p><p>- Comments within each rubric category- Comments in a discussion format upload of </p><p>critique - Tracked changes and comments in document</p></li><li><p>10/29/2014</p><p>21</p><p>EvaluationTools</p><p>Study2Design The 38 courses with peer assessments spread across </p><p>all masters degree programs served as the data pool for the study.</p><p> Random term was selected from 8 terms in 2013. From the total courses with peer reviews, 25% of the </p><p>courses were randomly selected. From the selected courses, 25% of the sections in </p><p>the identified term were randomly chosen. From each identified section, 25% of the students </p><p>were randomly chosen. </p></li><li><p>10/29/2014</p><p>22</p><p>Study2DesignCurriculum&amp;Instruction</p><p>EducationalLeadership</p><p>EducationalLeadership(TX)</p><p>Core C&amp;IESL/Bilingual</p><p>Totals</p><p>Year 6.3.2013 6.3.2013 6.3.2013 6.3.2013 6.3.2013Term(18) 4 4 4 4 4</p><p>Total#ofCoursesPeerAssessments</p><p>38</p><p>RandomlySelectedCourses(25%of38totalcourses=10)</p><p>1.CI53332.CI5423</p><p>1.EL5033 1.EDX51232.ELX56233.EDX5153</p><p>1.ED51532.LIT5203</p><p>1.BE50132.BL5063(0studentsthis</p><p>term)</p><p>10</p><p>NumberofSectionsPerSelectedCourse(25%oftotalsectionsperselectedcourse)</p><p>1(32)1(33)</p><p>1(31) 1(2)1(1)1(15)</p><p>1(29)1(34)</p><p>1(29)0(0)</p><p>206</p><p>NumberofStudentsRandomlySelected(25%oftotalperrandomlyselected</p><p>section)</p><p>88</p><p>8 114</p><p>79</p><p>70</p><p>53</p><p>NumberofSelectedStudents</p><p>ParticipatinginPeerAssessmentPer</p><p>Course</p><p>7/88/8</p><p>8/8 0/10/13/4</p><p>7/78/9</p><p>5/70</p><p>46</p><p>GuidingQuestion1 What is the level of student involvement in the peer </p><p>review assessment process? </p></li><li><p>10/29/2014</p><p>23</p><p>PeerReviewLevelofInvolvement(87%)</p><p>n =53studentsinsample46</p><p>7</p><p>05101520253035404550</p><p>Participant NonParticipant</p><p>PeerReviewersCompletingScoringRubric(85%)</p><p>n=46participantsofthe53sampled</p><p>39</p><p>7 7</p><p>0</p><p>510</p><p>1520</p><p>253035</p><p>4045</p><p>CompletedRubric DidNotCompleteRubric</p><p>NonParticipants</p></li><li><p>10/29/2014</p><p>24</p><p>PeerReviewersUsingTrackedChanges(24%)</p><p>n=46participantsofthe53sampled</p><p>11</p><p>35</p><p>7</p><p>0</p><p>5</p><p>10</p><p>15</p><p>20</p><p>25</p><p>30</p><p>35</p><p>40</p><p>TrackedChanges DidNotTrack NonParticipants</p><p>PeerReviewersMakingRubricRelatedComments(96%)</p><p>n=46participantsofthe53sampled</p><p>44</p><p>27</p><p>05101520253035404550</p><p>MadeComment DidNotComment NonParticipant</p></li><li><p>10/29/2014</p><p>25</p><p>GuidingQuestion2 To what extent are students applying peer </p><p>feedback to improve their work prior to final submission?</p><p>StudentsApplyingPeerReviewFeedbackBeforeFinalSubmission(63%)</p><p>n=46participantsofthe53sampled</p><p>20</p><p>9</p><p>6</p><p>11</p><p>7</p><p>0</p><p>5</p><p>10</p><p>15</p><p>20</p><p>25</p><p>AppliedFeedback PartiallyAppliedFeedback</p><p>ReceivedAllPositiveFeedback</p><p>DidNotApplyFeedback</p><p>NonParticipants</p></li><li><p>10/29/2014</p><p>26</p><p>PeerReviewFeedbackCourse ObjectiveAlignedtoProgramOutcomes</p><p>Identifymethodstoevaluatelearnermotivationandcharacteristicsandcontextualfactorsthataffectlearnermotivation.(InstructionalProgram,LearningEnvironment)</p><p>CI5333 StudentEngagement AssignmentDescription</p><p>InthismodulesApplication,youwillexamineandreconstructanexistingcurriculumunitthroughthelensofstudentengagementandmotivation.Intheprocess,youwillapplytheARCSModelofMotivationandtheTARGETframework.</p><p>YoudidagreatjobdescribinghowtheARCSModelofMotivationisincorporatedintheunitandwhatspecificmodificationyouwouldmaketoimprovetheunit.Ithinkaddingconcreteoceananimalswouldincreasetheattentionofyourstudents.Ialsolikeyourfrogtasticwall.Ithinkitisagreatwaytodisplaystudentswork.</p><p>PeerReviewFeedbackCourse ObjectiveAlignedtoProgramOutcomes</p><p>Identifymethodstoevaluatelearnermotivationandcharacteristicsandcontextualfactorsthataffectlearnermotivation.(InstructionalProgram,LearningEnvironment)</p><p>CI5333 StudentEngagement AssignmentDescription</p><p>InthismodulesApplication,youwillexamineandreconstructanexistingcurriculumunitthroughthelensofstudentengagementandmotivation.Intheprocess,youwillapplytheARCSModelofMotivationandtheTARGETframework.</p><p>Ienjoyedreadingabouttheunityoudeveloped.IalsoteachWestwardExpansion,soitwasinterestingtoreadsomeoneelsesperspectiveonteachingthecontent.Ithinkyourunitdoesfocusonincreasingintrinsicmotivatingandengagingstudents.Yourpaperwaswellwritten,butIthinkinordertomakeyourpointsmorepowerfulitwouldbehelpfultoincorporatemoreresearch.Forexample,youcouldcitewhyyourchoiceofinstructionalactivitiesareengagingandmotivating.Finally,thereweresomeminorAPAerrors.Ihaveuploadedaneditedpaperwithmyformattingchangesinred.Thanks!</p></li><li><p>10/29/2014</p><p>27</p><p>PeerReviewFeedbackCourse ObjectiveAlignedtoProgramOutcomes</p><p>Explainthecharacteristicsofactionresearchandtheprocessforconductingaresearchproject(VisionforLearning,InstructionalProgram)</p><p>EDX5153ResearchMethods AssignmentDescription</p><p>InthismodulesApplication,youwilldescribethemethodologyyouwillusetocollectthedataforyourstudy.Yourmethodologyisyourdatacollectionplan.</p><p>Youreintroductiondefinitelyaddressesaneedforliteracyimprovementandverygooduseofdatatosupportyourneed.IwouldrecommendalittlemorespecificsinthequestionssuchasWhateffectivestrategiescanbeusedtoincreaseclassroomjournalwritingamonglowsocioeconomicAfricanAmericanstudents?Thejournalarticlesyouselectedprovidegoodinformationandaredirectlylinkedtoyourresearchquestions.Iwouldrecommendnarrowingyourfocus.Therearetoomanyvariablestokeeptrackofwiththreeseparatestrategies.Theuseofdataisagoodfoundation,perhapstyingthequestionstogethertoaddressasinglestrategywouldbemoremanageableforthetimeframeweareworkingwith.</p><p>PeerReviewFeedbackCourse ObjectiveAlignedtoProgramOutcomes</p><p>Describe formalandinformalas...</p></li></ul>

Recommended

View more >