Web-based Proficiency Exams and Mastery Learning in Mathematics

  • View
    42

  • Download
    0

Embed Size (px)

DESCRIPTION

Web-based Proficiency Exams and Mastery Learning in Mathematics. John Lindsay Orr University of Nebraska - Lincoln. Slides. http://www.math.unl.edu/~jorr/presentations. Goal. Answer the question: What is an Online Assessment Management System and why use it? Definitions Examples - PowerPoint PPT Presentation

Transcript

  • Web-based Proficiency Exams and Mastery Learning in Mathematics John Lindsay OrrUniversity of Nebraska - Lincoln

  • Slideshttp://www.math.unl.edu/~jorr/presentations

  • GoalAnswer the question:

    What is an Online Assessment Management System and why use it?

    DefinitionsExamplesCore FeaturesFive practical problems

  • Who Am I and Why Am I Here?

  • What is an OAMS?An Online Assessment Management System is web-based software to manage student assessment:

    Student: Step through questions, answer questions, getgrade report and feedback

    Instructor: Write questions, assemble assignments,review grades

    Varied question types and assignment types

  • ExamplesSome Online Assessment Management Systems include:

    Maplesofts MapleTAHorizon Wimbas EDU

    WebAssign from North Carolina State UWebWork from Rochester U

    Wileys WileyPlusPrentice Halls PHGradeAssist

  • Core FeaturesThe core features of an OAMS are:

    Automated grading, immediate feedback Range of question types Range of assignment policies (quiz, homework, test, etc.) Web interface for students to browse questions Grade reporting for student and instructor (grade book) Tools to create new questions Administrative tools (user management) Interface with other web applications (CMS)

  • Five Practical Problems1. Students dont always remember all the material from previous courses.

    2. Some students scrape through on partial credit who shouldnt.

    3. I have some students in my class who shouldnt be there.

    4. I know what I said but I dont know what they heard.

    5. My students dont all study as hard as Id like them to.

  • Five Practical Problems

  • Five Practical ProblemsStudents dont always remember all the material from previous courses

    Practice more the first timeLearn it until youve mastered itReview it until youre as good as you were (or better)

  • Five Practical ProblemsSome students scrape through on partial credit who shouldnt

    Only award credit once theyve mastered it

  • Five Practical ProblemsI have some students in my class who shouldnt be there

    Placement testing

  • Five Practical ProblemsI know what I said but I dont know what they heard

    Just in time teaching (www.jitt.org)Pre-class warm-ups with feedback loop

  • Five Practical ProblemsMy students dont all study as hard as Id like them to

    No pay/no play Increased time-on-task

  • Some Pedagogical Strategies1. Mastery Learning

    2. Just-in-time Learning

    3. Adaptive Learning

    4. Increased time-on-task

  • A False Choice

  • How Technology Can HelpIn addition to Automated grading and Immediate feedback

    an OAMS can provide 24/7 Access

    andRepeatable assignments

  • 24/7 Access

    Chart2

    9320

    2410

    152

    52

    60

    10

    40

    242

    7018

    10257

    92117

    85100

    91105

    10174

    113134

    124144

    95122

    72101

    104117

    10680

    11335

    12932

    11432

    10235

    All Others

    Testing Center

    Chart1

    93

    24

    15

    5

    6

    1

    4

    24

    70

    102

    92

    85

    91

    101

    113

    124

    95

    72

    104

    106

    113

    129

    114

    102

    data 1

    0:00-0:591,485932912:00

    1:00-1:594602421:00

    2:00-2:593491522:00

    3:00-3:5972513:00

    4:00-4:5989604:00

    5:00-5:5946105:00

    6:00-6:59104416:00

    7:00-7:594222467:00

    8:00-8:591,18970228:00

    9:00-9:591,917102269:00

    10:00-10:592,344922810:00

    11:00-11:592,248852011:00

    12:00-12:593,550913412:00

    13:00-13:593,760101341:00

    14:00-14:593,896113462:00

    15:00-15:593,604124343:00

    16:00-16:593,23395284:00

    17:00-17:592,28972225:00

    18:00-18:592,855104286:00

    19:00-19:592,462106387:00

    20:00-20:593,008113398:00

    21:00-21:593,502129359:00

    22:00-22:592,8811143510:00

    23:00-23:592,6231023011:00

    TOTAL48,3881,78546

    Chart 2

    20

    10

    2

    2

    0

    0

    0

    2

    18

    57

    117

    100

    105

    74

    134

    144

    122

    101

    117

    80

    35

    32

    32

    35

    data 2

    0:00-0:592,1532013

    1:00-1:59965104

    2:00-2:5932521

    3:00-3:5919720

    4:00-4:592100

    5:00-5:592300

    6:00-6:59000

    7:00-7:594821

    8:00-8:599681810

    9:00-9:593,4475727

    10:00-10:597,56311750

    11:00-11:597,80310036

    12:00-12:597,92210535

    13:00-13:595,5167446

    14:00-14:599,33713449

    15:00-15:599,86414452

    16:00-16:5911,68812260

    17:00-17:597,09310144

    18:00-18:598,31011743

    19:00-19:596,2148027

    20:00-20:592,3853520

    21:00-21:592,9883214

    22:00-22:592,1143223

    23:00-23:592,7283515

  • Repeatable AssignmentsEach student on each attempt gets a different set of questions.

    1. Permuted questions

    2. Algorithmic questions

  • Algorithmic Questions$a = range(4, 9); $k = decimal(2, rand(1, 2));$right = sum(x, 1, $a, x($a-x)e^(-$k x));$low = $right - 3; $high = $right + 3;

  • Case StudiesSome ways weve used OAMS at Nebraska:

    1. Gateway Exams 2. Placement Testing 3. Keller Plan 4. Homework

  • Gateway ExamsPrecalculus, Calculus 1, Calculus 2

    Require 80% fully correct for credit

    Credit is 5% of course total

    Daily retakes allowed for two weeks

    Proctored exams and unlimited practice

  • Placement TestingAll incoming students take diagnostic test Proctored from UNL or high school Diagnostic assessment algorithm

    (Maplesoft + MAA product)

  • Keller PlanSelf-paced introductory psychology course Students progress through units by mastering tests Tests mix multiple choice and essay Both machine and human feedback available

  • HomeworkIncreased time-on-task

    Publisher and home-grown content

  • Advantages/Disadvantages Standardizes courses+ Reduces grading work+ Reduces grading budget+ Enhances limited classroom time

    - Administration- Question creation

  • EDU at UNLBegan project in 1996 at UNL

    Developed Wiley Webtests and then eGrade 1.0 for John Wiley & Sons

    Developed EDU for Brownstone Learning (now Horizon Wimba)

    Became core of Maplesofts MapleTA

  • EDU at UNL > CoursesFall 2003

    170 active classes set up

    Sheet1

    Fall 1996Spring 1997Fall 1997Spring 1998Fall 1998Spring 1999Fall 1999Spring 2000

    Math 106Math 106Math 103EdPs 860Curr 359Chem 110Astr 103Astr 103

    Math 107Math 106Hist 201EdPs 859EdPs 860BioSci 230(X)Bios 202

    Math 107Math 102Hist 201Hist 171BioSci 301Chem 109

    Math 103Math 102Hist 201BioSci 418/818Chem 110

    Math 106Math 103Hist 220Chem 109EdPs 860

    Math 107Math 106Math 102Chem 110Geol 100

    Phys 211Math 107Math 103EdPs 859Geol 101

    Math 208Math 106Geol 101Hist 120

    Phys 211Math 107Hist 100Hist 171

    Phys 211HMath 208Hist 171Hist 201

    Psyc 296Phys 141Hist 201Lib 110

    Psyc 350Phys 151Math 102Math 102

    Psyc 461Phys 212HMath 103Math 103

    Psyc 296Math 104Math 104

    Psyc 461Math 106Math 106

    Math 106HMath 107

    Math 107Math 107H

    Math 203Math 203

    Math 208Math 208

    Phys 141Math 208H

    Phys 151Phys 212

    Phys 212Phys 212H

    Phys 212HPols 100

    PolS 100Psyc 181

    Psyc 181Psyc 350

    Psyc 268Psyc 463

    Psyc 350Psyc 910

    Psyc 463Stat 180

    Psyc 975Stat 380

    Thtr. Stud.

    &C&"Arial,Bold"&12Classes Served by Semester

    Sheet2

    Sheet3

  • EDU at UNL > DisciplinesAccountingCurr & InstructionPsychologyAg. MarketingEducational Psych.School of MusicAgro. & Hort.Electrical Eng.Veterinary ScienceAnthropology EntomologyLincoln High SchoolArchitectureFamily & Consumer ScienceAstronomyGeographyBiochemistryGeosciencesBiological SciencesHistoryBio Systems Eng.Industrial Eng.Children, Family & LawLibrariesChemistry MathematicsCivil Engineering Modern LanguagesCommunication StudiesNatural Resource SciencesComputer SciencePhysicsConst. ManagementPolitical Science

  • EDU at UNL > StudentsFall 2003

    Student accounts: 10,650 Active (5+ sess.): 7,075

    Spring 2004

    Student accounts: 8,863 Active (5+ sess.): 5,310

  • EDU at UNL > AssignmentsFall 2003

    Total assignments:277,148 Daily average: 2,500

    Spring 2004

    Total assignments:292,888 Daily average: 2,650

  • EDU at UNL > Value AddedFall 2003

    Total assignments:277,148GTA hourly rate: $20GTA assignments/hour: 10

    Cost of human grading: $554,296

  • OAMS for MathsWhats specific to math for online assessment?

    Grading formulas and expression on content; CAS

    Powerful question types

    Mathematical notation in display and entry

    LaTeX or WYSIWG question creation

    Sophisticated algorithms ( x+-2 )

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • Questions

  • DemosMapleTA 2.51

    MapleTA 3.0 (Beta)

    EDU at UNL

  • Goalhttp://www.math.unl.edu/~jorr/presentations