How to Fail a StudentHow to Fail a Student

Lisa M. Beardsley-Hardy, PhD, MPH, MBALisa M. Beardsley-Hardy, PhD, MPH, MBADirector of EducationDirector of Education

General Conference of Seventh-day General Conference of Seventh-day AdventistsAdventists

Mene, Mene, Tekel, ParsinMene, Mene, Tekel, Parsin

MeneMene : God has numbered the days of : God has numbered the days of your reign and brought it to an end. your reign and brought it to an end.

TekelTekel : You have been weighed on the : You have been weighed on the scales and found wanting. scales and found wanting.

PeresPeres : Your kingdom is divided and : Your kingdom is divided and given to the Medes and Persians.given to the Medes and Persians.““

(Dan 5:26-28 NIV)(Dan 5:26-28 NIV)

How to Fail a StudentHow to Fail a StudentAbsolute cut-off scoresAbsolute cut-off scores

Norm-referenced cut-off score Norm-referenced cut-off score (e.g. 1 SD below mean fails 15% (e.g. 1 SD below mean fails 15% of class independent of of class independent of competency)competency)

Criterion-referencedCriterion-referenced

How to Fail a StudentHow to Fail a StudentProvide experience with assessment Provide experience with assessment and feedback.and feedback.

Sample multiple times.Sample multiple times.Plan for leniency to be fair to Plan for leniency to be fair to other students (e.g., require final other students (e.g., require final paper in week 13 of semester) to paper in week 13 of semester) to allow grace.allow grace.

Use multiple methods (multi-trait, Use multiple methods (multi-trait, multi-method sampling).multi-method sampling).

Determine and document reliability.Determine and document reliability.

“Empirical rule” or 68-95-99

Nedelsky MethodNedelsky MethodDetermine minimally competent or Determine minimally competent or borderline borderline ““F-DF-D”” student student

Divide 1 by the number of Divide 1 by the number of answers from which the test answers from which the test taker has to guess. taker has to guess.

Sum expected scores and divide Sum expected scores and divide by number of questions on the by number of questions on the exam.exam.

Nedelsky exampleNedelsky example My music teacher thinks that Marian My music teacher thinks that Marian

Anderson sings ________any other Anderson sings ________any other contralto he has ever heard.contralto he has ever heard.

(A)(A) more well thanmore well than

(B)(B) better thanbetter than

(C)(C) the best ofthe best of

(D)(D) more better overmore better over

Eliminate A & D, guess between B & C = Eliminate A & D, guess between B & C = ½=.50½=.50

Modified Nedelsky MethodModified Nedelsky MethodDetermine minimally competent or Determine minimally competent or borderline borderline ““F-DF-D”” student student

Divide 2 by (2 + the number of Divide 2 by (2 + the number of answers) from which the test answers) from which the test taker has to guess. taker has to guess.

Sum expected scores and divide Sum expected scores and divide by number of questions on the by number of questions on the exam.exam.

Modified Angoff MethodModified Angoff MethodRequires a copy of the test or at least a Requires a copy of the test or at least a wide selection of items from the test item wide selection of items from the test item pool.pool.Asks each judge to Asks each judge to ““state the probability state the probability that the that the ‘‘minimally proficient personminimally proficient person ’’ would would answer each item correctlyanswer each item correctly”” or the number of or the number of borderline students out of 100 who would be borderline students out of 100 who would be expected to get the item correct.expected to get the item correct.Multiple rounds of ratings and discussion Multiple rounds of ratings and discussion are conducted.are conducted.Probabilities are then summed and the total Probabilities are then summed and the total sum = cut score.sum = cut score.The ratings and discussion must be conducted The ratings and discussion must be conducted separately for each cut score being separately for each cut score being determined.determined.Used in NAEP, CLEP and many state programs, Used in NAEP, CLEP and many state programs, particularly on tests that are all Multiple particularly on tests that are all Multiple Choice Choice (see www.collegeboard.com)(see www.collegeboard.com)

Konosuke MatsushitaKonosuke Matsushita

Matsushita Company (ticker MC)Matsushita Company (ticker MC)– PanasonicPanasonic

http://panasonic.net/corporate/http://panasonic.net/corporate/philosophy/code/04.htmlphilosophy/code/04.html

Evidence of Closing the LoopEvidence of Closing the LoopBroad discussion of outcomes dataBroad discussion of outcomes data

Discussion of implications of data and Discussion of implications of data and outline institutional learning agendaoutline institutional learning agenda

Document institutional learning in Document institutional learning in committee minutes, annual departmental committee minutes, annual departmental report, and reports to students, BT, report, and reports to students, BT, and accreditation bodiesand accreditation bodies

DiscussionDiscussionObservationsObservations

QuestionsQuestions

RecommendationsRecommendations

Next stepsNext steps

Faculty DevelopmentFaculty Development

Purchase or download Purchase or download A Primer A Primer on Setting Cut Scores on Tests on Setting Cut Scores on Tests of Educational Achievementof Educational Achievement from from

http://www.ets.org/Media/http://www.ets.org/Media/Research/pdf/Research/pdf/Cut_Scores_Primer.pdfCut_Scores_Primer.pdf