Learner Centered Assessment in Online Course in Online Courses ... Design anddeliverinstructionthatisresponsivetostudent ... learner-centered assessment:

Embed Size (px)

Text of Learner Centered Assessment in Online Course in Online Courses ... Design...

  • 10/29/2014

    1

    LearnerCenteredAssessmentinOnline

    CoursesUsingSelfEvaluationandPeerReviewtoDocumentStudentLearningOutcomes

    Shawntel Landry, Ed. D.Alana Sloan, Ed. D.

    Assessment InstituteOctober 21, 2014

    Agenda1. Background/Set the Stage2. Assessment System3. Learner-Centered Assessment4. Learner-Centered Assessment: Study 15. Learner-Centered Assessment: Study 26. Overall Findings and Continuous Improvement

  • 10/29/2014

    2

    BackgroundandSettheStage

    AmericanCollegeofEducation

    Regionally accredited graduate-level education college

    Located in Indianapolis, IN

    Programso 16 graduate certificateso 23 masters programso 1 specialist programo 1 doctoral program

  • 10/29/2014

    3

    SettingtheStage Moved in 2013 from a proprietary learning management

    system (LMS) to Canvas LMS Increased interactivity and opportunity for learner-centered

    assessment through more automated survey capabilities and peer review assessment

    Prior to 2013:- pre- and post- self-assessments were in Word documents, and data were analyzed by hand.

    - peer reviews could only be conducted in discussion boards.

    GuidingQuestions How can learner-centered assessment provide

    evidence of student mastery of learning outcomes?

    What measures can serve as rich data sources?

  • 10/29/2014

    4

    ResearchQuestionCan learner-centered assessment be used to provide evidence of student mastery of learning outcomes?

    Lets take a poll

    PollResults

  • 10/29/2014

    5

    AssessmentSystem

    AssessmentSystemFourComponents

    StudentLearningOutcomes

    Quantitative

    Qualitative

    Performance

    LearnerCentered

  • 10/29/2014

    6

    SLOAlignment

    AcademicOutcomes

    ProgramOutcomes

    CourseObjectives

    ModuleObjectives

    rubricssurveysinstitutionwide

    stateandnat.standardsdisciplinespecificstandards

    ProgramBlueprintandSLOAlignments

  • 10/29/2014

    7

    AssessmentAlignment Course Level

    o Course TOTS Alignment Map

    o Course Alignment Map

    Program Levelo Program Alignment Map

    Course Levelo Course Assessment Alignment

    Map (Signature Assessments, Peer Review)

    AmericanCollegeofEducationT2T/MATProgramAlignmentMap

    Revised2.14.12

    ProgramOutcomesCourseObjectives

    EDTT520 EDTT504 EDTT506 EDTT507EDTT508

    EDTT511

    EDTT502

    DL501

    EDTT513 EDTT516 EDTT501 EDTT515

    EDTT505

    %ofEmphasis

    1.Designanddeliverinstructionthatisresponsivetostudentdiversityanddifferencesandpromoteshighachievementforallstudents.

    15,16 1,2,3,4,5,6,7,8,9,11,12,13,14,16,

    17

    1,2,3,6,7,8,9,11,12,13,14,15,16,17,18

    3,7,12

    4,5,6,8,11

    1,9,12,13,15,16

    2,4,6,10

    3,4,6,7

    1,4,6,7,8,9,10,15

    5,6

    23%

    2.Usestandardsbased,datadrivendecisionmakingandintegrated,differentiatedinstructionthateffectivelyintegratesappropriatetechnologiesandtoolsandsupportsthelearningofallstudents.

    4,6,7,18,19,20,23

    7,11 12,18 2,4,8,9,10,13

    4,5,6,8,11

    2,3,4,6,7,8,17

    1,4,6,10

    1,2,3,4,5

    3,4 4,6,7,8,9,10,14,17,18,19,20,21

    3 2

    19%

    3.Explainthemultipleformsofassessment,anduseassessmentresultstoinforminstructionaldecisionmaking.

    2,13,14,21,22

    4 4,5 5,6,10,12,14

    3 8,10 11 7 1,2,7 22,23 1,2,3,4,5,6,7

    1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22

    4

    19%

    4.Constructwellorganized,positivelymanaged,andsafelearningenvironmentsthatpositivelyimpactinstructionaleffectivenessandstudentachievement.

    2,10,15 11 7 18 7,8 2,3,4,5

    5 2,3,11,12,13,14

    1,3,58%

    5.Validateparticipationinactivitiesthatpromotecontinuousprofessionalgrowth,inquiry,research,andcollaborationwithotherstoimprovestudentachievementandtomaintainthehighethicalstandardsoftheteachingprofession.

    3,8,9,24,25,26

    10,18 10 3,14 14 1,3,5,9,12,13,14

    6 1,2 2,3,5,16,25,26,27,28,29,30,31,32,33,

    34

    1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22

    4

    21%

    6.Applyknowledgeofscientificallybasedresearchtoplananddeliverinstructionincontentareas,includingreading,languagearts,mathematics,science,socialstudies,finearts,health,wellness,andphysicaleducation,forthepurposeofimprovingtheachievementofallstudents.

    1,5,10,11,12,17

    1,2,3,5,6,9,12,14,16,17

    1 1,2,9,10,11

    4,11 8 6

    9%

    LearnerCentered

    Assessment

  • 10/29/2014

    8

    LearnerCenteredAssessment

    An activity designed by the professor that yields comprehensive information for analyzing, discussing, and judging a learners performance on valued abilities and skills (p. 12)

    - Huba & Freed (2000)

    Lets take a poll

    PollResults

  • 10/29/2014

    9

    BenefitsofaLearnerCenteredApproach

    At the student level: Students become active participants rather than

    passive consumers of content (p. 903).- Bayat (2012)

    Learner-centered assessment:- helps students synthesize learning experiences.

    - Huba & Freed (2000)

    BenefitsofaLearnerCenteredApproach

    At the institutional level, learner-centered assessment:- promotes high expectations in the learning environment.

    - demonstrates respect for diverse talents and learning styles.

    - promotes coherence in providing data to improvecurriculum and guide revisions.

    - Huba & Freed (2000)

  • 10/29/2014

    10

    LearnerCenteredAssessment:TwoStudies

    1. Pre- and Post-Course Self-Evaluation to measure Student Learning Outcomes (SLOs)

    2. Peer Review Assessment to measure SLOs

    LearnerCenteredAssessment:Study1

    (SelfAssessment)

  • 10/29/2014

    11

    SelfAssessmentDefinitionsThe involvement of learners in making judgments about their own learning, particularly about their achievements and the outcomes of their learning

    - Boud & Falchikov (1989)

    Evaluations learners make about their current knowledge levels or increases in their knowledge levels in a particular domain.

    - Sitzman, Ely, Brown, & Bauer (2010)

    BenefitsofSelfAssessment

    Debate: Are self-assessment results more representative of cognitive or affective learning growth?

    Self-assessment can be used as a triangulation technique to supplement information (about student learning) acquired through direct measures.

    - Moskal (2010)

    Change in students psychological perspectives resulting in: Metacognitive knowledge Learning constructivism Development of self-efficacy

    - Basnet, Basson, Hobohm, & Cochrane (2012)

  • 10/29/2014

    12

    BenefitsofSelfAssessment

    Self-assessments of knowledge have a critical role in the learning process; learners benefit from having an accurate understanding of their knowledge levels.

    - Dunning et al. as cited in Sitzman et al. (2010)

    GuidingQuestions What data can we harvest from pre- and post-

    course self-evaluations? How can we graphically show this data?

  • 10/29/2014

    13

    ResearchQuestion Can self-assessment provide evidence of mastery of

    student learning outcomes?

    Study1Design Three programs M. Ed. in Curriculum and

    Instruction, Educational Leadership, Educational Technology

    Random term selected from 8 terms in 2013 From the term, 10% of the total number of courses

    randomly selected from a numbered course list Pre- and Post- Self-Evaluations collected and

    analyzed from all student sections of selected courses

  • 10/29/2014

    14

    Study1DesignCurriculum&Instruction

    EducationalLeadership EducationalTechnology

    Year 6.3.2013 11.18.2013 4.22.2013Term(18) 4 8 3NumberofCoursesPerTermPerProgram

    8 5 3

    NumbersandTitlesofCoursesPerTerm

    1.CI50332.CI53133.CI53334.CI53535.CI53936.CI54237.CI54538.CI5503

    1.EL50332.EL56233.EL56534.EL57535.EL5903

    1.ET50132.ET50233.ET5033

    RandomlySelectedCourseNumbers(10%oftotalcoursesperselectedterm)

    5.CI5393 2.EL5623 3.ET5033

    NumberofSectionsPerSelectedCourse

    2(56) 5(153) 1(30)

    NumberofStudentsCompletingPre andPostSelfEvaluations

    Pre 35Post 29

    Pre 118Post 101

    Pre 22Post 21

    DataComparisons:C&I

  • 10/29/2014

    15

    DataComparisons:EL

    DataComparisons:ET

  • 10/29/2014

    16

    Findings In all selected courses and across programs, students self-

    reported significant growth in the acquisition of student learning outcomes when pre- and post- results were compared.

    Across programs, post-course results showed an overwhelming majority of students rating their knowledge level as a 3 or 4.

    4-Point Likert Scale1. I am unfamiliar with this concept.2. I have heard of this but need to learn more.3. I am fairly confident in my knowledge of this and can

    implement this on my own.4. I can teach or consult with someone else to help them

    implement this.

    Findings:Item1PerceivedGrowthComparison

    0 20 40 60 80 100 120

    Curriculum&Instruction

    EducationalLeadership

    EducationalTechnology

    PostPre

  • 10/29/2014

    17

    LearnerCenteredAssessment:Study2

    (PeerAssessment)

    PeerAssessmentA method in which students evaluate each others work based on set criteria

    - Falchikov (2001) & Gouli et al. (2006)

  • 10/29/2014

    18

    BenefitsofPeerAssessment

    Peer evaluations help students who are being evaluated by adding insight of people (fellow students) with different perspectives than their professor.

    The act of evaluating peer work can strengthen critical thinkin