Evaluating the Effectiveness

of Your Training:

Kirkpatrick’s 4 Levels

Left Click to move to next slide and begin next audio.

PRESS the F5 key to begin the slideshow.

The Principles and Process of Good Training Development & Delivery

1. Training Needs Assessment2. Establish Learning Objectives3. Content4. Delivery 5. Evaluation

Is the training program a quality program?

Does it follow...

National Institute of Corrections Model:Instructional Theory Into Practice (ITIP)

Training CycleNeeds

Assessment

Evaluation

Evaluation

Deliver Training

Pilot Test

Set Performance Objectives

Revise

Design Training

Train Instructors

Assessment of Training Needs

• In order to be effective, a training must be based on the needs of the trainees, and the “needs” or goals of the organization.

• Identify “The Gap” -- between what skills employees actually have --and what they should be

• Who should determine need for training? Management, trainees themselves, co-workers, specialists in the field,…

Needs Assessment Sample Questionnaire

Rate your competence in each of the following areas giving a 1 to the lowest and a 5 to the highest. Give a different number to each.

Rating Juvenile Careworker Skill

Establishing clear expectations

Balancing positive & negative feedback

Giving immediate and appropriate consequences

How to use your facility’s behavior management plan

5

3

1

2

Establishing Learning Objectives

• Skills-based or competency-based objectives

• Knowledge-Based objectives: more likely to include what the learner will understand or be able to explain

• Attitudes: individual attitudes needed to fully perform

Learning Objectives

Competency goals should be:

• Behaviorally anchored • Observable • Forward-thinking • Discrete• User-friendly

Example: Supervisor frequently gives accurate and specific feedback to subordinates about their performance.

Hackett

Training Content• Does content meet best practices

established by leading organizations in the field of juvenile confinement? (NIC, NJDA, CJCA, NCCHC)

• Does the content cover the needs of staff?

• Does it accurately reflect local policies and procedures?

• Is the content timely? Does it reflect current research and theory?

Training Delivery

• Meets the needs of all learning styles – active/kinetic, visual, social, audio

• Covers all parts of the learning cycle – perceiving, reflecting, applying, creating

• Organized presentation• Positive learning environment• Empathetic, knowledgeable,

professional trainers • Et cetera

Training Evaluation: Does the training actually work?

• Does the training leave your staff feel more motivated, more confident, more knowledgeable?

• Does the training affect the way staff do their job? How are youth affected by the change?

• Is it causing the change in staff’s performance that we want? To the degree that we want?

Evaluating Training Effectiveness:

Kirkpatrick’s 4 levels

Level 1: Reaction

Level 2: Learning

Level 3: Behavior Change

Level 4: Results

Level 1 Evaluation: Reaction Training Evaluation• Did you like this training?• Was the material relevant to your job?• Was the trainer knowledgeable?• What do you think was the most

important thing you learned?• Are the handouts helpful?• Comments: _______________

A positive reaction does not guarantee learning, but a negative reaction almost certainly reduces its possibility.

Level 2 Evaluation: Learning

55

60

65

70

75

Pre-Test Post-Test

Comparison of Test Performance

Other methods can be less formal, such as trainer observations during practice, team assessments, or self-assessments.

Level 3 Evaluation – Behavior Change on the Job

Guidelines:

1. Use a control group if practical

2. Evaluate both before and after the training if possible

3. Survey and/or interview people who can observe the behavior

4. Get 100 percent response or a random sampling

5. Repeat the evaluation at appropriate times including after allowing a time lapse

For trainee: How much has the program improved the working relationship between you and your co-workers?

Large extent Some

extentNo

change Made worse

For supervisor: Since the training, how well does the trainee do in making decisions?Much better

little better The

same Little

worseMuch worse

For subordinates & trainees: Since the training, how much much does she/he ask for your opinion or support to your ideas?More often

15 trainees

Little more

5 trainees

The same

0 trainees

Less often

0

5 staff 15 staff

10 staff 0

Level 3 Evaluation continued: Example questions

Level 4 Evaluation: Organizational Impact

• Measures broad-based changes in areas that are important to the agency as a whole, such as: – decreased legal costs– Safety improvements– Increased arrests

• Compare measures before and after training.

• Complicated and Expensive

Sample results of a Level 4 Evaluation

Change in use of physical restraints as measure of Behavior Management training

$0

$20

$40

$60

Number of incidents