1Presented by Tom Hoverman

A Methodology for Measuring Results

2Presented by Tom Hoverman

Why Evaluate Learning?

A Strategic Approach to Learning

4 Levels of Learning Evaluation

A Learning Evaluation Methodology

Barriers to Evaluating Learning

3Presented by Tom Hoverman

“Never mistake activity for achievement.”

- John Wooden, UCLA Basketball Coach

4Presented by Tom Hoverman

Learning is a critical business process. The value it brings to the organization needs to be evaluated with the same rigor applied to other critical business processes.

Organizations that evaluate and continuously improve their learning programs enhance both individual performance and business results.

5Presented by Tom Hoverman

1. Partner with stakeholders

2. Agree on the relevant business needs

3. Target specific performance improvement goals

4. Identify enabling new knowledge and/or skills

5. Develop specific learning objectives

6. Design and deliver learning activities

7. Evaluate the costs and benefits related to learning

8. Use this analysis to plan future investments in learning

6Presented by Tom Hoverman

Measurement of Results

Planning & Design

Evaluation of Results from Learning

Classroom & Online Learning Activities

Learning Objectives

Performance Goals

7Presented by Tom Hoverman

Evaluation Supports Strategy

Learner Reaction Level 1

Growth in Knowledge and/or Skills Level 2

Performance Level 3

Business Impact Level 4

Strategy Frames Evaluation

8Presented by Tom Hoverman

Learner Reaction Level 1

Growth in Knowledge and/or Skills Level 2

Performance Level 3

Business Impact Level 4

Did this learning measurably impact business results?

Have the learners applied their new knowledge and/or skill?

Did the learning result in a measurable increase in knowledge and/or skill?

Did the learners find the learning activity worthwhile and do they

intend to apply it?

9Presented by Tom Hoverman

Content Development

Instructional Design

Learning

Training Delivery

Investments

InvestmentsReturns

Expanded Knowledge

Business Results

Improved Performance

Opportunity Costs

New Skills

Financial Return

10Presented by Tom Hoverman

Learner Reaction

Focus of Measurements

Specific level of satisfaction and reaction to the learning solution as it is delivered to the participants.

Actions to Evaluate Learning

Measure employees’ initial reaction, satisfaction and planned action derived from learning event (examples: post-course survey assessing quality of course.)

11Presented by Tom Hoverman

Growth in Knowledge and/or Skills

Focus of MeasurementsSpecific knowledge, skill(s), and/or attitude(s) to be developed/acquired

by the participants.

Actions to Evaluate LearningAnalyze learning with pre- and post-learning tests,

simulations and instructor evaluations.

12Presented by Tom Hoverman

Focus of MeasurementsParticipants’ behavior changes as the knowledge and skills are applied in

the work setting.

Actions to Evaluate LearningAnalyze knowledge and skills gained over time to see if on-the-job

behaviors changed as result of learning event.

Gather data on how employees are using new skills.

Identify barriers and enablers, and isolate the effects of the program from other factors that can influence behaviors.

Performance

13Presented by Tom Hoverman

Focus of MeasurementsSpecific business measures that will change as a result of the

participants’ application of the training.

Business Impact

Actions to Evaluate LearningMeasure the business impact/benefits.

Determine the monetary value of the measured change.

Measure intangible benefits (e.g. increased job satisfaction among participants)

14Presented by Tom Hoverman

Life cycle of the solution

Relevance to organizational strategy

Program costs

Learner audience size

Level of visibility

Level of management interest

15Presented by Tom Hoverman

1. Make the business case for evaluating learning

2. Relate the learning curriculum directly to business goals

3. Choose the appropriate level of evaluation for each course

4. Design evaluation to reflect learning objectives

5. Integrate learning evaluation into existing processes

16Presented by Tom Hoverman

“They’re just trying to make us look bad” the perception of evaluation as a fault-finding exercise

“It’s not worth the bother” the concern that evaluation is a poor investment

“We don’t know how” the issue of having access to the right expertise

17Presented by Tom Hoverman

“What gets measured gets done.”

- Peter Drucker, Management Guru