21
Ego-Defensive and Ego-Promotional Behavior in Socially Desirable Responding and Self-Assessed Job Performance: A Faking Study Proposal J. Peter Leeds December 14, 2005

Ego Defensive and Ego Prmotional Responding

Embed Size (px)

DESCRIPTION

my doctoral dissertation presentation on socially desirable responding in bio data and self-assessed job performance

Citation preview

Page 1: Ego Defensive and Ego Prmotional Responding

Ego-Defensive and Ego-Promotional Behavior in Socially Desirable Responding and Self-Assessed Job Performance: A Faking Study Proposal

J. Peter Leeds

December 14, 2005

Page 2: Ego Defensive and Ego Prmotional Responding

• "Vices and virtues are of a strange nature, for the more we have, the fewer we think we have"  (Alexander Pope, English Poet, 1688-1744)

Introduction

• Why it’s important to study faking as egoist responding• What I hope to gain for this research• Some background literature (do people fake and does it matter, problems with SDR

scales)• Introduce egoistic responding and underlying theory• Connect egoistic responding with self-appraised performance• Present hypotheses• Methodology, and research design, measures and analyses• A second study involving the construct validation of the proposed SDR measure will

be presented

Page 3: Ego Defensive and Ego Prmotional Responding

Rationale for the study

• Non-cognitive testing is “big” and much hangs in the balance.

• People can and do fake on such measures

• Faking distorts validity, score hierarchy, and measurement properties of tests.

• Understanding faking components and correlates has value in controlling faking.

• The utility of decomposing faking as denial of vices and an over claimed virtues has received equivocal results in the literature.

Page 4: Ego Defensive and Ego Prmotional Responding

Literature

• What is faking? – A subclass of SDR where one deliberately responds to non-cognitive measures in ways that reflect favorably on oneself.

• What is Socially Desirable Responding (SDR)? – A response bias where one tends to gives inaccurate responses that reflect positively on one self. 3 types of

• SDR according to Paulhus (1984):

1. impression management

2. subtle self-deception

3. self-deceptive denial• People can and do fake

Ones, Viswesvaran, and Korbin (1995) found that respondents are able to increase their scores on personality tests by an average of .5 of a standard deviations.

McDaniel, Douglas, and Snell (1997) found wide spread faking among job seekers. Griffith et al. (in press) found .60 standard deviation in an within subject

applicant sample design

Page 5: Ego Defensive and Ego Prmotional Responding

Does faking matter?

• Faking degrades criterion related validity - Christiansen, Goffin, Johnston, & Rothstein, (1994) showed that as the proportion of fakers increased from 0 to 25%, the observed validity dropped from .34 to .19.

• Adjusting scores based on faking produces increments to validity – Although controversial, Army Research Institute reported improving criterion validity using faking score adjustment with military samples and Lao (2001) fund that controlling for the effects of faking increased the relationship between conscientiousness and job performance. Be careful here, this is one of only a few studies that find this…report the others Ones et al…

• Faking changes the rank order of candidates - White and Kilcullen (1997) observed that 29% of high SDR responders and only 6% of the low SDR responders would have passed a reasonably set cut-score.

• Faking distorts measurement properties of non-cognitive tests - Griffith (1997) found method a method bias contamination disrupted the measurement properties for a group instructed to fake on a battery of personality assessments. Frei (1997) found significant differences in the factor loadings, error variances, and factor intercorrelations .

Page 6: Ego Defensive and Ego Prmotional Responding

Detecting faking and the problems with SDR scales

They don’t work – reasons cited include:

• the transparency of faking items

• high false negative and false positive classification rates

• poor face validity

• correlation to latent trait

• poor intercorrelation among faking scales

• validated with fake good instructional sets

Page 7: Ego Defensive and Ego Prmotional Responding

Egoistic Responding

All SDR test items either pose a vice and ask for denial or pose a virtue and ask for self-attribution

Examples:

I never return money I have found. Vice, prompting ego-defensive SDR

I never swear. Virtue, prompting ego-promotional SDR

Equivocal research findings –

Dires (1964) Schultz (1998) Jacobson et al (1977) support whileMillham (1974) and Phillips (2004) refute

Both denial of vice and self-attribution of virtue have at least one thing in common – Both serve the interests of the self

Page 8: Ego Defensive and Ego Prmotional Responding

Theoretical basis

Prospect theory (Kahneman & Tversky, 1979) – Undervalue gains and over-valuing losses. Acknowledging vices is a greater “loss” to ones ego than claiming virtues a “gain” to the ego.

Theory on depressive realism – the tendency of mildly depressed persons to have more accurate self perceptions than those who are not depressed. Predicts that those who claim vices and deny virtues (self-depreciating) hold lower and more accurate impressions of their own performance. McCrae and Costa, (1983) found that those scoring higher on a depression scale also tend to score low on impression management scales

Self-affirmation theory - the desire to maintain the perceived worth and integrity of the self is the fundamental goal of a self-regulating system (Steele, 1988).

Page 9: Ego Defensive and Ego Prmotional Responding

Summary of theoretical basis

Prospect theory indicates that defensive faking will be more prevalent than promotional faking.

Depressive realism suggests that ego deflated persons will show less faking than others.

Self-affirmation theory posits the basic motivations underlying all defensive and promotional motivations.

By couching faking in terms of egoistic responding we are able to not only predict people’s tendency to promote/defend their ego on tests but also in their self appraised job performance.

Page 10: Ego Defensive and Ego Prmotional Responding

Self Appraisal and Egoistic responding

• The Harris and Schaubroeck (1988) meta analysis showed that self and supervisor performance appraisals differ substantially (r = .35 uncorrected) with subordinates having much higher self appraisals.

• Atkins and Wood (2002) used assessment center performance to validate 360-degree performance ratings. Results showed 1) that supervisor-obtained performance appraisals were much lower than the self-obtained performance appraisals offered by those who overestimate their actual assessment center performance. Found that braggarts get marked down but not as much as supplicants.

• Both faking and self-appraisal are subject to the same egoistic responding predicated by self affirmation theory.

Page 11: Ego Defensive and Ego Prmotional Responding

Hypotheses

I. Prospect Theory would predict that there will be significantly higher mean level of SDR scores (more SDR) on ego defensive items than the ego promotional items.

II. Prior research suggests that those highest on SDR will evidence more self-supervisor performance disparity than those who are lowest on SDR.

III. Self-affirming theory suggests that there will be a significant and strong correlation between SDR scale scores and the self-supervisor discrepancy scores.

IV. Inferences from Atkins and Wood (2002) suggest that there will be no correlation between supervisor assessed performance ratings and SDR scores since 1) those who are high ego promoters will tend to receive lower and discrepant supervisor ratings (-r), 2) those who are low ego defenders/promoters (i.e., prone to modest self depreciation) will also receive low, but non-discrepant supervisor ratings (+r) and 3) those who are high ego defenders will not be detectable (r = 0).

V. Self Affirmation theory suggests that there will be a significant positive correlation between SDR and self assessed performance since the same egoist responding will take place.

VI. Bio data (CLIMB) score adjustments based on ego promotional and/or ego defensive responding scores will improve the criterion validity of the bio data (CLIMB) measure significantly.

Page 12: Ego Defensive and Ego Prmotional Responding

Study 1

Methodology

Research Design

The design will:

Compare the amount of SDR detected by ego defensive and ego promotional SDR items.

Examine relationships between different types of SDR items and different sources of job performance data (self versus supervisor).

Examine and propose causal explanations for the degree to which different sources of performance appraisal ratings are affected by different types of SDR behavior.

Participants

A sample of 300-500 U.S. Army and Navy civilian supervisor- subordinate pairs drawn from around the world.

Page 13: Ego Defensive and Ego Prmotional Responding

Apparatus

1. Composite SDR Scale

A 20-item measure of SDR was constructed from items selected from:

1. The impression management scale of version 6 of Paulhus’s Balanced Inventory of Desirable Responding, BIDR (1991)

2. The Marlowe-Crowne Scale (1960).

Only positively phrased items were selected to avid the confusion of negations (e.g., I never tell lies” is a negation of “I always tell the truth”)

The SDR scale presents 10 positively phrased items representing virtues and 10 positively phrased items representing vices - Appendix A.

A separate SDR scale validation study will be proposed and is labeled as Study 2 below.

Page 14: Ego Defensive and Ego Prmotional Responding

Apparatus Continued

2. Civilian Leader Improvement Battery (CLIMB)

A web-based, criterion validated biographical data questionnaire containing 72 items with 8 scales measuring:

1.Work Motivation, 2.Cognitive Flexibility, 3.Peer Leadership, 4.Stress Tolerance, 5.Social Perceptiveness, 6.Team Orientation, 7) 7.Socially Desirable Responding, and 8.Self-Efficacy - Appendix B

Page 15: Ego Defensive and Ego Prmotional Responding

Apparatus Continued

3. The Leadership Effectiveness Inventory (LEI)

• A web-based self and supervisor assessment

• LEI asks both the employee and the supervisor to provide ratings on a 1-5 scale on:

1.how well he/she currently performs 105 leadership-related asks 2.how competent he/she needs to be to perform these tasks for promotion.

• Each task is linked to one of 27 leadership competencies - Appendix E

Page 16: Ego Defensive and Ego Prmotional Responding

Procedure

The CLIMB 20-item SDR composite, and LEI 180-degree performance appraisal will be web-enabled.

Army and Navy civilian employees will be notified via an internal marketing campaign of the opportunity to participate in the supervisory skill assessment battery.

Data will be collected and maintained by a contractor

After about 6 months, 300 to 500 score sets will be available for analysis.

Page 17: Ego Defensive and Ego Prmotional Responding

Analysis

1 A T-test will determine if SDR scores (more SDR) on ego defensive items are significantly greater than the SDR scores on ego promotional items. (Hypothesis I)

2 A correlation coefficient will be used to measure the relationship between SDR and self-supervisor performance appraisal discrepancy. (Hypothesis II and IIII)

3 F-Test will determine whether self-supervisor performance appraisal discrepancy scores (DV) are different among faking types (e.g., self depreciators, ego promoters, ego-protectors) the IV. (Hypothesis II)

4 A correlation coefficient will determine if there is significant relationship between supervisor assessed performance ratings and SDR scores. (Hypothesis IV)

5 A correlation coefficient will determine whether there is a significant relationship between self assessed performance and SDR scores. (Hypothesis V)

6 A step-wise regression analysis will be used to assess whether bio data (CLIMB) score adjustments based on SDR scores will improve criterion validity of the bio data (CLIMB) measure significantly. (Hypothesis VI)

Page 18: Ego Defensive and Ego Prmotional Responding

Analysis Continued

6. Polynomial Regression: In order to examine the 3-dimensional relationship between 1) SDR (y-axis), 2) supervisor assessed job performance (x-axis), and 3) self-assessed job performance (z-axis) polynomial regression will be used. A fitted regression plane will show the complex relationship. A separate polynomial regression will be run for ego-protective, ego-promotional, and combined SDR scores.

Example

The data might show that the relationship between SDR and supervisor assessed performance may change depending on the level of self-assessed performance. Conversely, the data may show that as self-assessed performance and SDR changes depending on the level of Supervisor-assessed performance.

Page 19: Ego Defensive and Ego Prmotional Responding

Study 2Validation Study of the Composite SDR Scale (Griffith, Rutkowski, Gujar, Yoshita, &

Steelman, (2005)

• Measures of SDR don’t work well (Snell & Sydell, 1999 and Ellingson, Sackett, & Smith, 2001).

• Correlation between faked and honest responses r = .50 (Griffith et al, 2000) • A construct (criterion? We are trying to predict a behavior) validation study of

the 20-item composite measure of social desirability is proposed.

Methodology

Participants Two groups of between to 50 to 100 introductory psychology students

Measures The proposed 27-item measure is a combination of the 20-item composite drawn

from 1) the Balanced Inventory of Desirable Responding, BRDI (Paulhus, 1991) and

the Crowne-Marlow (1960) scale and2) 2) the 7 items developed by the Army Research Institute.

Page 20: Ego Defensive and Ego Prmotional Responding

Study 2 Continued

Procedure

The “deception first” condition – Visiting consulting firm accepting applications for an attractive job. Subjects take the 27-item faking measure and are debriefed. Participants asked to complete the measure again, this time honestly.

The “deception last” scenario – Visiting consulting firm requesting norming data for new 27-item measure. Subjects take the test. Two weeks later re-approach students with good news (we have an attractive job) and bad news (we lost first test scores). Subjects retake measure and are debriefed.

The design counterbalances the deception sequence (deception first and last) to account for resentful responding and provides two measures of the scales construct validity.

Page 21: Ego Defensive and Ego Prmotional Responding

Study 2 Continued

Analysis

A correlational analysis will be used to examine the relationship between the scales scores obtained under the motivated condition and under the honest condition.

Faking difference scores will be calculated showing the mean change in subjects’ scores from those obtained under the motivated condition and those obtained under the honest condition. Observed mean over maximum mean shows the sensitivity of the scale to faking.

Correlate SD scores to the change scores between applicant and honest

Manipulation Check

Since participants will be led to believe that there is an available job they will be asked to rate their interest in that job on a 5-point scale. The magnitude of the mean of this item across subjects will indicate the confidence we can place in the manipulation.