Evaluating Training ProgramsThe Four LevelsBased on the work of Dr. Donald L. Kirkpatrick & Dr. Myron A. Eighmy *Muhammad AwabM&E Specialist, TESP SecretariatHigher Education Commission, Islamabad

ObjectivesUpon completion of this presentation you will be able to:

State why evaluation of programs is critical to you and your organization.Apply Kirkpatricks four levels of evaluation to your programs.*

Why Evaluate?Determine what instructional strategies workDetermine the effectiveness of the program design

Learn the needs for improvement*

Four Levels of EvaluationKirkpatrick ModelDuring program evaluation Level ReactionLevel LearningPost program evaluationLevel BehaviorLevel Results*

Level : Reaction A customer satisfaction measureWere the participants pleased with the programPerception: if they learned anythingLikelihood of applying the contentEffectiveness of particular strategiesEffectiveness of the packaging of the course*

Level 1: ExamplesQuantitative:

The contents of this training were relevant and applicable to my work.(Please circle a number)

Strongly DisagreeDisagree Dont KnowAgreeStrongly Agree12345

*

Level 1: ExamplesQualitative: The best part of this program wasThe one thing that could be improved the most ..

*

Level : LearningWhat did the participants learn in the program?What exactly did the participant learn and not learn?Instrument: Pretest & Posttest FormsRequires developing specific learning objectives to be evaluated.

*

Level 2: ExamplesDevelop a written exam based on the desired learning objectives.Use the exam as a pretestProvide participants with a worksheet/activity sheet that will allow for tracking during the session.Emphasize and repeat key learning points during the session.Use the pretest exam as a posttest exam.Compute the posttest-pretest gain on the exam.*

What makes a good test?The only valid test questions emerge from the learning objectives.Consider writing main objectives and supporting objectives.Test questions usually test supporting objectives.Ask more than one question on each objective.

*

ExampleMatch the following to the choices below___ Reaction Level ___ Learning LevelA. Changes in performance at workB. Participant satisfactionC. Organizational ImprovementD. What the participant learned in class*

Pre & Post Test Analysis*

Level : BehaviorHow the training affects performance?The extent to which change in behavior occurred.Was the learning transferred from the classroom to the real world?*

Level 3: Guidelines for EvaluatingMeasure on a before/after basisAllow time for behavior change (3-6 months)Survey or interview one or more who are in the best position to see change.The participant/learnerThe supervisor/mentorSubordinates or peersOthers familiar with the participants actions. *

Level 3: InstrumentsObservationSurvey or InterviewParticipant and/or othersPerformance benchmarksBefore and afterControl group*

Level : ResultsImpact of education and training on the organization or community.The final results that occurred as a result of training.The ROI for training.*

Level 4: ExamplesHow did the training save costsDid work output increaseWas there a change in the quality of workDid the social condition improveDid the individual create an impact on the communityIs there evidence that the organization or community has changed.*

Level 4: Guidelines for EvaluatingMeasure before and afterAllow time (6-12 months or even more) for change to take placeRepeat at appropriate timesUse a control group if practicalConsider cost vs. benefits of doing Level FourRemember, other factors can affect resultsBe satisfied with Evidence if Proof is not possible.

*

Any Questions?Thank you*

*