1

Unit 2: Performance Assessment & Measurement

Traditional Performance Appraisal

Performance Assessment

Performance Measurement

2

Unit 2 Unit exam over study objectives (27 points) one

week from tonight Muchinsky: traditional performance appraisal Daniels chapter: behavioral measurement & performance

matrix Pampino et al. article - great example of use of performance

assessment, multiple component intervention, and social validity assessment. Only problem is short-term nature of the study

Iwata et al. article - antecedents vs. antecedents + consequences

Pinpointing exercise (8 points) due Wed Description of performance measurement project

3

Pinpointing exercise For a job you have had or are familiar with, identify at

least 3-4 key performance measures (behaviors or products of behavior)

Measures should be objective that could be targeted in a PM intervention

If possible, include 1-2 measures that could be improved and indicate “improvement” after those

Format Typed Put job title/position at the top Just bullet your measures unless you feel they need to be

explained

4

Performance Measurement Project Proposal due Monday, Oct. 15 (35 pts) Final project due Monday, Dec. 10 (35 pts) See syllabus for policy on lateness Two options

Development and implement a measurement system for a minimum of 4 weeks

If measures already exist and you can obtain those measures from archive for 4 weeks, implement an intervention

(Final project due Monday, 12/3 if grade before ME2)

5

Project description You may work with in groups or by yourself

If you work in a group, each individual will receive the same grade - can’t separate individual contributions

Job One or more workers: one is OK Full-time or part-time A friend, relative, significant other, another student,

someone you supervise: only restriction - not one for yourself

Business or educational setting (i.e., PSY 3600 lab instructors or teachers, PSY 1000 assistants, servers)

Advise against secretaries and programmers

6

Some basic rules related to employment jeopardy The emphasis will be positive - I will not accept a

project in which the objective is to document poor performance

You must inform the worker or workers whose performance will be measured; if possible, give employees an option

If the supervisor or manager will see the data, you must insure that he/she will respond to it positively, not negatively If supervisor responds negatively and you and I cannot

resolve that, then I will have you terminate the project

7

HSIRB issues

Research: a systematic investigation that contributes to generalizable knowledge

Projects in this class do not constitute research, however, they could (more later)

HSIRB class registration of projects I have registered the class with the HSIRB and

gotten initial approval for the projects After the proposal, but before you start the project I

will send titles to HSIRB for final approval You cannot implement your project until I get

this final approval!!

8

Why aren’t the projects research?

The data will only be used in-class or internally by the organization: thus do not contribute to generalizable knowledge If you use this as a pilot for your thesis or present the data

publicly, then the project becomes research: Why else would you do that if the data don’t contribute to generalizable knowledge?

Confidentiality issues arise when projects are publicly presented: cannot violate a participant’s right to confidentiality

You must submit a full HSIRB protocol if you use this asa pilot study for your thesis or present it publicly!!!

(and before you can do that, on-line training; sample of a full protocol in course pack)

9

Why aren’t the projects research? (cont.) In OBM, you are being invited by management to

implement a management system Organizations have the right to implement and evaluate

management systems as part of their normal business practice They can require all employees to participate in that system

so you do not have to get consent from the employees (you do need a letter of consent from management)

You are not a researcher who is implementing the intervention, you are implementing or evaluating the system on behalf of management (consultant, agent) With the caveat that data will only be used internally

(important issues, related to projects, theses, questions?)

10

Ethical questions to ask yourself

Can the employees be harmed by the data you collect - are you placing them in employment jeopardy? Group vs. individual data? (Ok to use individual data) If individual data, can the individuals be identified?

Are you using the least restrictive and least intrusive procedure that is effective? (aversiveness) Publicly posted individual data with identifiers vs. group data

or individual data that are coded Confidentiality of the employees

Should not discuss performance of individuals with identifying information with anyone outside the organization

Proposal and final report to me: do NOT include the name of the organization or employees

11

Performance Measures

Objective System may cover:

2-3 measures that need to be improved Measures that target only the critical performances Measures that cover all of the major responsibilities

Must be repetitive - daily or weekly measures Verifiable - if you ask the worker to self-record you

must develop a system to verify accuracy Strongly recommend accomplishments rather than

behaviors Behaviors require you to be observe the person while

working - very labor and time intensive (biggest problem in past)

(in syllabus, mention a book by Abernathy provides measures)

12

Measurement system and forms

What will be measured and why the measure is important

Who will measure? When will measurement take place? How frequently will measures be taken? If self-report, how will the data be verified? Include the measurement form Sample graph or data summary - how are

you going to present the data?

13

Proposal format and grading Both format and grading are discussed in the project

description A sample proposal and sample final project reports are

included in the course pack in this unit Strongly recommend you get started now - proposals

are due in 3 1/2 weeks Yes, I will take them early, if you want to get started

early

QUESTIONS??

14

Traditional Performance Appraisal: Muchinsky

SO1: Characteristics of performance appraisals that affect the way the courts rule

Performance appraisals are subject to federal and state EEO laws, and more and more cases are being filed.

Protected classes under Title VII?

African Americans, Hispanics, Asians, Native Americans,and females.

(on click)

15

SO1 Five characteristics that affect the way courts rule1. Should be objective (NFE, no brainer)2. Should be job related, preferably based on a

written job analysis (NFE, no brainer)3. Should be based on behaviors, not traits

Research shows no difference between trait-based forms and behavior-based forms, but courts think they are better. (problem is still subjectivity)

4. Performance should be under the control of the employee

5. Should relate to specific functions of the job, not global assessments

(#2 could be a problem for a behavioral measurement system if used for personnel decisions and no job analysis)

16

Common types of appraisal methods (NFE)

Graphic rating scales, most common Employee comparison methods

Evaluate employees by comparing them against each other rather than against a standard

Behavioral checklists and forms

17

SO3A: Employee comparisons vs. graphic rating scales

Advantage of employee comparison over graphic rating scales? Forces variance or variability into the ratings With graphic scales, not uncommon for a

supervisor to rate employees pretty much the same: everyone is a 5 or a 6 on a 7 point scale, making it difficult to make personnel decisions.

18

SO3B: What’s the problem with employee comparisons?

May create a false impression that differences between individuals are actually larger than they are

Six employees, rank ordered1. Jan2. Shakira3. Paul4. Mike5. Susan6. John

Actual performancedifference betweenJan and Mike or evenSusan may be verysmall, yet look big.Can affect salaryincreases, promotions,etc.

19

Dissatisfaction with performance appraisal

Companies tend to switch frequently from one type of form to another Recall performance appraisal was the third-ranked topic

addressed in JAP IBM abandoned employee ranking in favor of graphic

scales, then returned to ranking a few years ago Made the front page of the Wall Street Journal: ranking was

THE way to go

Why do you think IBM may have made the switch?

(nothing to do with reliability or validity)

20

SO4: Behavioral checklists Behavioral checklists and scales are assumed to be

more accurate because they are less vague, but they are not.

Why? Still are based on subjective judgments Completed once a year

That is also why we should not consider them adequate from a behavioral perspective Objective measures of behavior/performance Over time as behavior/performance occurs on the job

(aubrey daniels, problems, do it 1/2 as often)

21

SO6: Main problem with performance appraisals (NFE)

Inflation of ratings Main assumption about the cause of errors

Design of the form (hence why so much time and money is spent redesigning forms - “design out” rating errors

Lack of knowledge on the part of supervisors/managers - training

Rarely are the consequences for accurate ratings examined or considered (which leads to SO7)

(Important lead-in to SO7, Director of Personnel, WMU, ranks among the worst problems she has - fire/good ratings)

22

SO7: Consequences for inflated ratings (learn 3, name of principle for 4) No rewards for accuracy; no or few sanctions

for inaccurate ratings (how does the organization know if the ratings are accurate or inaccurate - a problem)

Most common reason - high ratings are necessary to get salary increases, promotions and other rewards for employees

Ratings received reflect their competence as a supervisor/manager

23

4th consequence

Negative evaluations result in defensive reactions and hostile reactions from employees

Principle of behavior?

Straightforward avoidance:In past: Bad rating ––––> SP aversive social interaction CS ––––––> CR stress

Current: Good rating –––––> Sr-, no aversive social interaction No CS to elicit CR of stress(on click)

24

SO8 Self-assessments

SO8A: Main problem? Inflation: we think we are better than we are Engineer example:

92 engineers rated their performance in comparison to other engineers. On average, each engineer rated his performance as better than 75% of the other engineers. Quite a feat to have 100% of your employees in the top 25%!

25

SO8 Self-assessments, cont. SO8B: Thornton - little agreement between supervisory and

self-assessments. What are the very important implications of this from a behavioral perspective?In behavior analysis we always strive to make rewards contingent upon performance. The better the performance, the more rewards.If, however, employees believe they are performing better than the supervisor believes, then the supervisor will not give them the rewards that they feel they should be getting. That is, employees will not believe that rewards are truly contingent upon their performance. Likely to hurt performance and cause dissatisfaction.

26

SOs 9&10: Peer assessments Peer assessments are very accurate - high reliability

and high validity Reliability - different people rate the employee the same way Validity - related to job performance

Main problem? Acceptance by employees - we don’t like to do this

WMU faculty merit system: Union vs. management Not enough money to go around; many just split $$

How would you like to be given the responsibility for evaluating your peers - the other students advised by your faculty advisor - and have rewards (i.e.,assistantships, grades in classes, opportunities for practica and projects) based on that?

(we may not know how well we are performing, but we know how well our peers are performing!)

27

SO11: 360 feedback: hot, hot, hot A manager/supervisor

Rates himself Is rated by his manager/supervisor Is rated by his subordinates

SO11A: What percentage of managers saw themselves as others saw them?

Only 10%!

SO11B: Overrating was most noteworty on what scale?

People

(NFE, but why is this good news for us?)

28

SO12: Credibility & Power - Fisher and Taylor Credibility: What influences credibility the most

from a behavioral perspective? (assuming manager has expertise)

Measurement system and its objectivity Judi Komaki - work sampling separates highly effective

managers from ineffective ones (actually objectively sampling and looking at work)

If the supervisor is not evaluating performance accurately, the feedback becomes meaningless

Daniels “In God we trust, all else bring data!!”

(important - ilgen, fisher & taylor, the extent to which people will change their performance when given fb from supv depends upon)

29

SO12B: Power and feedback Power: The extent to which a supervisor has control

over valued rewards Why does power influence the extent to which an

employee will be influenced by a supervisor’s feedback? (this is the question)

Feedback is not a basic principle of behavior. It is just a stimulus. It will be a neutral stimulus unless it is paired with valued consequences. When it affects behavior, it most likely does so as an SD or Sr (or an analog). But, in both cases to become an SD or Sr, it must be paired with valued consequences. Thus, if feedback is not paired with valued consequences (the supv. has no control over them), it will not affect behavior.

(excellent point from a behavioral perspective, students have trouble with this, a nice behavioral analysis)

30

For your entertainment only: Actual statements

His men would follow him anywhere, but out of morbid curiosity

I would not allow this employee to breed Works well when under constant supervision and

cornered like a rat in a trap He would be out of his depth in a parking lot puddle This young lady has delusions of adequacy This employee is depriving a village somewhere of

an idiot

31

End Part 1

Any questions so far?