Learning and Performance through InnovationLearning and Performance through Innovation1

RetrospectiveRetrospectiveAnalysis:Analysis:

A PrimerA Primer

Jeff M. AllenJeff M. Allen

Learning and Performance through InnovationLearning and Performance through Innovation2

When is the last time you learned a

newnew research

design?

Learning and Performance through InnovationLearning and Performance through Innovation3

Retrospective PretestRetrospective Pretest

Campbell and Stanley (1963) offered retrospective pretest as a tool to enhance experimental designs.

Used and promoted in the field of behavioral science research, since the 1950s as a moderator for the threat of response-shift bias(post-then, pre-post, then)

Learning and Performance through InnovationLearning and Performance through Innovation4

Solution: Retrospective PretestSolution: Retrospective Pretest

Howard (1980) began prescribing this research tool as a remedy for response-shift bias.

Distinguishes itself from the traditional pretest by its relationship to the intervention.

Learning and Performance through InnovationLearning and Performance through Innovation5

When and HowWhen and How

Learning and Performance through InnovationLearning and Performance through Innovation6

When does it work?When does it work?

Learners don’t know … what they don’t know

There is insufficient time for pre-test.

There as been no benchmarking performed.

Learning and Performance through InnovationLearning and Performance through Innovation7

Why does it work?Why does it work?

Administered post-intervention, asking the subjects to recall behavior

Can provide a more accurate measure of actual learning

Can provide more differentiation in learning (instrument scaling)

Learning and Performance through InnovationLearning and Performance through Innovation8

Early ExamplesEarly Examples

1951 Effects of racially mixed housing

on prejudice

1956 Measurement of fear

Learning and Performance through InnovationLearning and Performance through Innovation9

Strengths LimitationsStrengths Limitations

Detects response-shift effect

Extends posttest only designs

Can curb threats to history, selective mortality, and shift in subject perceptions

Draws on a subject’s memory

Best used when added to other designs

Inappropriate utilization for skill measurement

Learning and Performance through InnovationLearning and Performance through Innovation10

Retrospective PretestRetrospective Pretest Law Firm

Learning and Performance through InnovationLearning and Performance through Innovation11

Retrospective PretestRetrospective Pretest Professional Development

Learning and Performance through InnovationLearning and Performance through Innovation12

Benchmark

Learning and Performance through InnovationLearning and Performance through Innovation13

ScalingScalingPrePre

PostPost

ThenThen

NowNow

Learning and Performance through InnovationLearning and Performance through Innovation14

RetrospectiveRetrospectiveAnalysis:Analysis:

A PrimerA Primer

Jeff M. AllenJeff M. Allen