Chapter 4 2Training for Organizations
Research Skills
Review:One minute elevator speeches:
Why do a needs assessment? Why do an evaluation?
Chapter 4 3Training for Organizations
Selected Methods
The observation method The interview method The survey method Testing
Chapter 4 4Training for Organizations
A Step-by-Step Approach
Step 1. Identify needed data Step 2. Determine who has the
answers to your questions Step 3. Design the data collection
instrument determine major points, questions, and
sequence of the questions develop a coding or tabulating method to
analyze findings
Chapter 4 5Training for Organizations
A Step-by-Step Approach (cont.) Step 4. Have “experts” review the
draft instrument Step 5. Field test the instrument,
then distribute it to or interview targets
Step 6. Analyze and interpret responses
Chapter 4 6Training for Organizations
The Questions
Categorical questionsHow many people do you supervise?
Likert-type questions On a scale of 1 to 7….. Open ended questions
Chapter 4 7Training for Organizations
The Observation Method
Types of variables descriptive inferential evaluative
Chapter 4 8Training for Organizations
The Observation Method
Recording observational data elapsed time recording frequency recording interval recording continuous recording
Chapter 4 9Training for Organizations
The Observation Form
Group work: Develop an observation form you
could use as a guide in observing a newly hired training instructor.
Chapter 4 11Training for Organizations
Interview Guides
Structured questions
“How many performance appraisals did you conduct last month?”
Open questions
“When conducting performance appraisals, what were your biggest challenges?”
Chapter 4 12Training for Organizations
Preparing for the Interview
1. What do you need to know? 2. Who has the answers?
Chapter 4 13Training for Organizations
Choices at the Design Stage How many people to interview? Interview one-on-one or in small
groups? In person or by telephone (video
conferencing?) Location of interview (privacy,
comfort, interruptions)
Chapter 4 14Training for Organizations
Preparation for the Interview Will interviewees need to show you
something, such as their use of technology?
Audiotape? (questions of confidentiality)
Need for repeated access? (and relationship-building)
Potential problems (e.g., getting them to talk; getting them to stop talking!)
Chapter 4 15Training for Organizations
Conducting the Interview
Put the interviewee at ease Explain why you are there & why the
interviewee was chosen Estimate how long the interview will take Explain what you will do with responsesnote--Questions should be designed to help the
client clarify issues rather than provide information for the interview.
Chapter 4 16Training for Organizations
The Interview Strategy
Listening is more than hearing
One study reported that we spend 70% of our day communicating: writing 9% reading 16% talking 30% listening 45%
Chapter 4 17Training for Organizations
Stages of Interviewing
plan (who, what, when, where) prepare (template-based questions) set the stage (“partner” with
interviewee”) collect data (keep in mind the
purpose of the interview) close the interview (summary, thanks) follow-up (copy of results?)
Chapter 4 18Training for Organizations
Preparing for the Interview
What do you need to know? Who has the answers?
Chapter 4 19Training for Organizations
Choices at the Design Stage
How many people need to be interviewed?
Interview one-on-one or in small groups? Should the interview be in person or by
telephone? (videoconferencing?!) Where should the interview be held? How long should the interview be? How should data be recorded?
Chapter 4 20Training for Organizations
Choices (cont.)
Will interviewees need to show you something, such as their use of a technology?
Audiotape? (questions of confidentiality)
Need for repeated access? (and relationship-building)
Potential problems (e.g., getting them to talk; getting them to stop talking!)
Chapter 4 21Training for Organizations
Stages of Interviewing
plan (who, what, when, where) prepare (template-based questions) set the stage (“partner” with
interviewee”) collect data (keep in mind the
purpose of the interview) close the interview (summary, thanks) follow-up (copy of results?)
Chapter 4 22Training for Organizations
The Survey Method
The method has a long history…population counts and surveys of crop production (Why? Taxes!)
Sociologists first linked data collection instruments with statistical procedures for analyzing their data
Chapter 4 23Training for Organizations
Designing a Questionnaire
Questionnaire Template Directions Body Ending Demographic Data
Chapter 4 24Training for Organizations
The Mail Questionnaire
Advantages Low cost Greater
anonymity Considered
answers Accessibility
(wider coverage)
Disadvantages Requires simple
questions No opportunity to
probe No control over
who responds Often low
response rate
Chapter 4 25Training for Organizations
Group Assignment
To assess computing needs of students, write two:
categorical questions Likert-style questions open-ended questions
Chapter 4 26Training for Organizations
Tests
Measure how well someone does a specific task or what they know about given content
Must be valid and reliable
Chapter 4 27Training for Organizations
Types of Tests
Pretests (for placement purposes) Formative (evaluate learning
progress) Diagnostic (identify learning
problems) Summative (determine that
learning occurred)
Chapter 4 28Training for Organizations
Creating and Using Tests
1. Record test items 2. Review and edit test items
Use a table of specifications to ensure coverage and match with learning objectives
3. Arrange the items in the test and prepare directions for the test
4. Administer and score the test
Chapter 4 29Training for Organizations
Experimental Methods
Used in training environments to provide evidence of the usefulness of a specific training intervention. In short, is one learning method better than another, and if so, to what degree?
Chapter 4 30Training for Organizations
Using Experimental Designs Post-test Only Design
X O1
Pre-test, Post-test DesignO1 X O2
Chapter 4 31Training for Organizations
Two-group Pre-test, Post-test DesignO1XaO2
O1XbO2
Four-group Design O1XaO2
O1 O2
XaO2
O2
Chapter 4 33Training for Organizations
Review
The importance of a good data collection instrument
How to write good questions Important instrument guide design
elements The importance of planning
Chapter 4 34Training for Organizations
Exercise:
Break into groups againThen, using the sample interview
guide one person play the role of the interviewer, one the interviewee
Those watching should use the check sheet provided