Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC

7/29/2019 Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC

1/21

Negative Psychological Effects of SelectionMethods: Construct Formulation and an

Empirical Investigation into an Assessment Center

Neil Anderson*University of Amsterdam

Vicki GoltsiMediterranean College, Athens

This paper defines and formulates the construct of negative psychological effects (NPEs)of selection and assessment methods upon applicants. The results of an empirical studyinto NPEs in an operational assessment center are also reported. Applicants (n5107)completed measures at three timepoints: Time 1 (just before the AC), Time 2 (immediatelyafter the AC but before outcome decisions were known), and Time 3 (6 months after theAC). Both accepted and rejected candidates completed all three measures, which included

self-esteem, mental health, positive and negative affect, and career exploration behavior.No evidence of NPEs was found for rejected candidates despite significant between-groupdifferences on feedback reactions level items. Interestingly, well-being and positive affectdeclined slightly for successful candidates at subsequent measurement compared againstbaseline Time 1 norms. Reactions level outcomes did, however, differ significantlybetween accepted and rejected candidates, with the latter rating feedback dimensions farless favorably. These results are discussed in relation to future research into thepsychological impact of selection procedures upon candidates and the need for appliedpsychologists and HR practitioners to demonstrate that assessment methods do notexhibit NPEs upon applicants.

E xamination of applicant reactions to selectionmethods has been a rapidly burgeoning area ofresearch interest over recent years, whose findings have

implications for the practical design of assessment and

selection procedures by organizations. Several comprehen-

sive reviews of this research conducted in different

countries internationally have been published and are

now available (e.g., Anderson, Born, & Cunningham-

Snell, 2001; Chan & Schmitt, 2004; Imus & Ryan, 2005;

Ryan & Ployhart, 2000; Polyhart & Harold, 2004;

Schmitt & Chan, 1999). However, much of this researchhas concerned immediate-level reactions and preference

perceptions of applicants to different predictors, leaving

open to question the longer-term impacts and outcomes of

applicant exposure to specific selection methods (Bauer,

Maertz, Dolen, & Campion, 1998; Gilliland & Steiner,

2001). The unequivocal consensus across this body of

findings is that applicants express clear preferences for

job-relevant, transparent, and equitably administered

methods, but that this proliferation of studies has

failed to address centrally important questions of the

longer-term impact of selection methods upon candidate

decision making, attitudes to the organization, or indeed,

important issues of applicant psychological health and

well-being.

Reactions level research, while important and having

obvious implications for the design and conduct of

organizational selection procedure, can clearly only shedlight upon perceptual responses and preference evaluations

by applicants. As noted by Iles and Robertson (1997)

and Anderson (2005), this level of outcome research

can only present a picture of applicant responses that

relates to immediate-level reactive responses. The trigger

effects of these reactions to candidate decision making

in selection remain a pointedly under-researched area,

as do important questions over the impacts, both pragmatic

and psychological, of selection methods upon applicants.

The focus of the current paper is on the latter the negative

*Address for correspondence: Neil Anderson, University of Amster-

dam Business School, Roeterstraat 11, 1018 WB Amsterdam, The

Netherlands. E-mail: [email protected]

INTERNATIONAL JOURNAL OF SELECTION AND ASSESSMENT VOLUME 14 NUMBER 3 SEPTEMBER 2006

236r 2006 The AuthorsJournal compilation r 2006 Blackwell Publishing Ltd, 9600 Garsington Road,

Oxford, OX4 2DQ, UK and 350 Main St, Malden, MA 02148, USA


2/21

(and positive) psychological effects upon applicants of

undergoing a managerial assessment center.

This paper has two aims: (i) to provide an operational

definition and conceptual formulation for the construct of

negative psychological effects (NPEs) caused by expo-

sure to selection methods, and (ii) to conduct an

exploratory, longitudinal study into the potential NPEs of

a managerial entry-level assessment center. We also aim to

clarify the diffuse and widening conceptual space coveredby studies into applicant reactions by arguing that our

understanding of these outcomes will only be improved by

definitional clarity of the outcome variables under scrutiny.

Finally, from a methodological point of view, the results

from the empirical study reported here present novel and

disconfirmatory findings that exposure to a multi-day

assessment center caused no discernible NPEs among

rejected candidates. Effect size statistics are reported,

showing both a notable lack of NPEs over time upon

unsuccessful candidates, but also an intriguing lack of

improvement in self-esteem, psychological well-being, and

positive affect among successful applicants.

NPEs and Positive Psychological Effects(PPEs) of Selection Methods

Toward a Working Definition of NPEs

The concept of NPEs was first proposed by Anderson

(2004). Although emphasizing the importance of research

into the possibility that selection methods may have an

unintentionally negative impact upon rejected candidates,

the construct was not originally formulated in detail or a

working definition proposed. Here, we rectify this by

proposing that NPEs are most appropriately defined asfollows:

Declines in applicant psychological well-being, general

mental health, or core self-esteem that are inveterate,

measureable, and statistically demonstrable, and that occur

as a result of exposure to rejection decisions, violations of

applicant rights, or unprofessional feedback given to

applicants by recruiters, at any stage during organizational

selection or promotion assessment procedures.

There are four main aspects of this definition. First, in

accordance with the wider literatures in IWO psychology

on mental health at work, stress, and job-related burnout,NPEs relate to longer-term declines in important aspects of

applicant mental health, psychological well-being, or core

self-esteem. Why these referent constructs? All are centrally

important aspects of the overall well-being of the indivi-

dual and can be argued to be fundamental concepts

underlying mental health (e.g., Judge & Bono, 2001; Hart

& Cooper, 2002). All are longstanding constructs of

research interest in applied psychology, with literally

thousands of primary studies and many meta-analyses

having been published into their antecedent causes and

consequent effects. In accordance with Judge and Bono

(2001), we therefore view NPEs as being at the interface of

applicant mental health and core self-esteem, with NPE

being a composite construct that encapsulates mental

health and self-esteem outcomes relevant to applicant

psychological well-being in its widest sense.

Second, NPEs relate to causal effects that are inveterate,

measureable, and statistically demonstrable. The Shorter

Oxford English Dictionary defines the term inveterate as . . .firmly established by long continuance; deep-rooted;

obstinate. Thus, we define NPEs as relating to longer-term

effects that are robust and deep-rooted, but of course not

necessarily either permanent in their duration or debilitating

in their effects sufficient to satisfy Diagnostic and statistical

manual of mental disorders-Fourth Edition (DSM-IV)

criteria for on-going personality disorders (American Psy-

chiatric Association, 1994). This acuity level is of course

rather debateable. That is, the level of impact needed to

satisfy the definition could be variously argued to be much

lower (e.g., a short term and minor drop in mood states or

positiveness), or much higher (e.g., acutely negative anddispositional changes that are relatively permanent). Rather,

in common with Fletcher (1991), Schuler and Fruhner

(1993), and Iles and Robertson (1997), we envisage NPEs

as being present for several weeks and months after receipt of

a negative selection decision. Thus, such declines in well-

being and/or core self-esteem need to be measureable, that is,

to register on widely accepted and psychometrically devel-

oped instruments, and to be statistically demonstrable. Both

of these requirements need to be fulfilled for NPEs against an

organizations selection procedure to be demonstrated prima

facie, even before other definitional criteria that we propose

can be evaluated (specifically, causality). It can also be stated

that NPEs need to be distinguishable from immediate levelreactions, preference ratings, and applicant reactions to

feedback (see Iles & Robertson, 1997; Robertson, Iles,

Gratton, & Sharpley, 1991; Ryan & Ployhart, 2000). The

latter are short-term applicant perceptual reactions to

selection methods; NPEs refer to longer-term psychological

impacts that result from being exposed to such methods.

Indeed, distinguishingthese two levelsof analysis in applicant

reactions research immediate-level reactions and longer-

term NPEs is a key contribution of the present paper.

Third, we propose that although NPEs are more likely,

as a result, of receiving reject outcome decisions at any

stage during recruitment and selection procedures, they canalso occur where the candidate is subsequently accepted

but has been treated in demonstrably unprofessional or

unambiguously abusive ways by recruiters (Derous, Born,

& De Witte, 2004). Here, it is possible that even accepted

candidates could suffer NPEs if the procedural treatment of

the organization has been so extreme, so biased, or so

personally abusive as to trigger such perceptions of

procedural or distributive injustice or deeper-rooted

psychological impacts (Gilliland & Steiner, 2001). Anec-

dotal evidence of poor treatment in selection situations

NPES OF ASSESSMENT CENTERS 237

r 2006 The AuthorsJournal compilationr Blackwell Publishing Ltd. 2006 Volume 14 Number 3 September 2006


3/21

abounds, so we purposely include the possibility in our

operational definition that NPEs will not necessarily be

restricted to rejected applicants alone.

Finally, for NPEs to hold up as identifiable outcomes

from exposure to selection predictors, there is an onus of

proof of causality of exposure to the selection method itself

as the primary cause. Threats to causal inference need to be

examined and measured in that rejected applicants could

also have been exposed to other extraneous life events overthe time period of interest. Although this definition

acknowledges that NPEs can occur at any stage in

recruitment and selection, it is most likely that they will

emerge from methods that are of a longer duration, that

involve applicant preparation, that are job-relevant and

face valid, and that the results from which cannot easily be

externally attributed by applicants (Anderson, 2004; Chan

& Schmitt, 2004). Such effects are entirely plausible in

external selection procedures, but perhaps even more so in

internal assessment procedures for promotion where both

accepted and rejected applicants remain in the employ of

the organization (Fletcher, 1991). This raises the prospectthat NPEs can have knock-on effects to job performance,

both in the case of unsuccessful internal candidates who

remain in their existing job function and for successful

applicants who are given inappropriately negative feed-

back or who feel that their expectations of procedural

justice have in some way been violated. Thus, NPEs may

ultimately have considerable and quantifiable costs for an

organization in terms of poorer levels of job performance,

that is, NPEs will have a substantial and negative utility

value, whose exact costs to an organization may remain

unknown and unrecognized.

Toward a Working Definition of PositivePsychological Effects (PPEs)

Counter to these arguments, it is feasible to suggest that far

more positive effects might be observed where applicants

obtain successful outcome decisions or where candidates

receive positive feedback after taking part in a particular

selection method or procedure. Here, the opposite of NPEs

that can be hypothesized may occur, that is, PPEs. As a

direct, polemic opposite of NPEs, PPEs can be defined as

follows:

Increases in applicant psychological well-being, general

mental health, or core self-esteem that are inveterate,

measureable, and statistically demonstrable, and that occur

as a result of exposure to acceptance decisions, perceived

respecting of applicant rights, or complementary feedback

given to applicants by recruiters, at any stage during

organizational selection or promotion assessment proce-

dures.

Again, this definition supposes that such changes need

to be relatively enduring and measurable through typical

pre- and post-study designs. Moreover, it is likely that

perhaps such changes in psychological well-being and

mental health will be more commonly determined than

changes in core self-esteem as a dispositional, relatively

stable personality characteristic (e.g., Judge & Bono,

2001). Nevertheless, it is possible that where an applicant

is going through several selection procedures concurrently

(as is often the case), where applicants know others who

have been unsuccessful, and where they receive multiple

positive decisions and/or feedback comments on theircapabilities, that this results in enhanced core self-esteem at

least temporarily. The prospect for PPEs for successful

candidates is therefore noted and this tentative definition

advanced as an opposite set of outcome conditions to those

proposed for NPEs.

To summarize, these working definitions of NPEs and

PPEs are proposed to guide and provoke future research

into this neglected topic area. Both are intended to build

upon and extend earlier work by Anderson (2004) and to

provide a basic conceptual formulation for psychological

effect research and, importantly, to distinguish this level of

analysis from that of earlier research into applicantreactions and positiveness ratings of selection method

and feedback perceptions favorability.

Past Research into NPEs in Selection

Appendix A provides a summary of published studies into

applicant NPEs conducted since 1989 onwards. Studies

were identified from a comprehensive search and review of

the literature undertaken by the present authors. For the

sake of brevity, we used several rules of thumb to decide

whether or not a study should be included in this appendix.

Initially, the study must meet key aspects of our definitionof NPEs proposed above, that is, it must have included

outcome variables that relate to applicant mental health,

well-being, or core self-esteem. Studies that for instance

examined applicant reactions to selection methods alone

(e.g., Horvath, Ryan, & Stierwalt, 2000; Lievens, De

Corte, & Brysse, 2003) were not included in this table (for

such a comprehensive table covering reactions-level

studies, see Ryan & Ployhart, 2000). Furthermore, we

only included studies in this appendix if they examined

effects longitudinally during selection procedures, not just

immediate pre- and post-test effects of reactions to

cognitive ability tests and other measures (e.g., Macan,Avedon, Paese, & Smith, 1994). In this regard, past studies

have often measured key variables using such pre- and

post-method designs, but with notably few studies in-

corporating a longer-term outcome measurement time-

period some weeks or months following selection

decisions. Finally, we include studies relating to different

selection methods (not just assessment centers) in Appen-

dix A on the grounds that relatively little research has been

conducted across all predictors to date, and so a

comprehensive summary is warranted.

238 NEIL ANDERSON AND VICKI GOLTSI

International Journal of Selection and Assessmentr 2006 The Authors

Journal compilationr Blackwell Publishing Ltd. 2006


4/21

Overviewing this existing body of research and counter

to what might be expected, there are relatively few studies

that have so far examined aspects of psychological impact

caused by selection procedures. This appendix summarizes

the findings of the fourteen published studies we located

over this 16-year period (19892004, inclusive), and covers

all predictor methods, not just assessment centers. Far less

research has examined NPEs then than might have been

anticipated, even in comparison with the growth of studiesinto applicant reactions over recent years (Salgado, 1999;

Salgado, Viswesvaran, & Ones, 2002). Despite the relative

paucity of empirical studies, more research has been

conducted very recently (five studies were published in

2004 alone: Derous, Born, & De Witte; Bauer, Truxillo,

Paronto, Campion, & Weekley; van Vianen, Taris,

Scholten, & Schinkel; Schmitt, Oswald, Kim, Gillespie,

& Ramsay; and, Schinkel, van Dierendonck, & Anderson),

and important findings emerge from this emergent stream

of research. A long-established focus of interest has been

the impact of cognitive ability and GMA testing upon

applicants (Gilliland, 1994; Lounsbury, Bobrow, & Jensen,1989; Ployhart & Ryan, 1997; Ployhart & Ryan, 1998;

Ployhart, Ryan, & Bennett, 1999; Schinkel et al., 2004;

Schmit & Ryan, 1992; Smither, Reilly, Millsap, Pearlman,

& Stoffey, 1993; van Vianen et al., 2004). Consistent race

differences in attitudes to GMA testing, links between

cognitive tests and favorable attitudes toward the organi-

zations selection procedure, recommendation and offer

acceptance intentions by applicants, and impacts upon self-

assessed performance, fairness, anxiety, and candidate

affective well-being have all been reported across these

studies. Notably fewer studies could be located that have

examined the psychological impacts of several other

popular selection methods, however, including structuredinterviews, personality inventories, work sample tests, or

situational judgment tests.

Assessment centers, despite their increasing popularity as

final-stage predictors for selection in several countries (e.g.,

Lievens & Klimoski, 2001), have similarly received scant

research attention into their potential for NPEs. Only two

peer-reviewed studies could be located that have examined

such longer-term psychological impacts upon actual job

applicants, each with directly contradictory findings (see also

Schuler & Fruhner, 1993). Robertson etal. (1991) conducted

a cross-sectional study into the impacts of three methods

biodata, situational interviews, and assessment centers atdiffering stages in the selection procedure. They found no

significant effects for assessment center experience or

outcome decision upon subsequent measures of applicant

self-esteem or psychological well-being. Fletcher (1991),

conversely, undertook theonly truly longitudinal study of AC

psychological impact, measuring applicant responses im-

mediately before, immediately after, and 6 months following

the AC itself. Significant declines in rejected applicants job

involvement and mood at work were reported, while

significant increases in successful applicants work ethic,

acquisitiveness, status aspiration, and job mastery were

found. Fletcher alsoreports significant maineffectdifferences

at Time 3 measures between successful and unsuccessful

applicants on measures of organizational commitment, need

for achievement, and psychological well-being. Note that this

AC was for an internal promotion procedure within a single

financial services organization where the unsuccessful

candidates continued to work alongside those who were

successful, and therefore, subsequently promoted. Despitethe laudible longitudinal element of this study design, the

sample was only 57 respondents,40 of whom were successful

and only 17 of whom were rejected for an accelerated

promotion program, an unusual ratio in comparison with

those normally found in assessment centers for selection.

Given that so few studies have been conducted into the

longer-term impact of ACs upon candidates, our under-

standing of these potential effects is at best inadequate. As

Fletcher (1991) correctly argues, exposure to a multi-day,

job-relevant, assessment procedure that is evaluated by

trained assessors who themselves are perceived as success-

ful organizational members, and where often quite detailedfeedback is given to candidates all serve as factors to

heighten the potential for NPEs against rejected applicants.

Likewise, we may hypothesize that successful completion

of such an interventive and psychologically demanding

method may well result in improvements in candidate

psychological well-being and self-esteem in having com-

peted successfully against many more candidates who

were subsequently rejected: such effects might be contrarily

termed Postitive Psychological Effects, or PPEs. Whether

this is the case or not, operational ACs for selection and

internal promotion present a critical case opportunity in

which to conduct research into such effects.

Method

Sample

A sample of 107 candidates at final stage assessment center

participated in the study. The candidates applied for a

highly prestigeous managerial-entry and graduate training

scheme within a leading multinational organization head-

quartered in the United Kingdom. The training scheme

lasted 3 years in total and included fixed-term postings to

various sectors of the business, all of which involvedperiodic attendance at trainee managerial courses and

leadership development events. Of this total of 107

participants, 53 (49.5%) were female and 54 (50.5%)

were male. This proportion of virtually equal males and

females at AC reflects the initial proportions of applicants

to this organizations selection procedure. Candidate age

ranged from 21 to 37, with a mean of 23.4 (SD52.6).

Fifty-seven percent of participants reported having past

experience of attending an assessment center, while the

modal figure for numbers of past ACs experienced was 2.




5/21

Of the 107 applicants, 28 were successful (26.2%) whereas

79 (73.8%) were rejected following completion of this AC.

Recruitment and Prescreening Procedure

Following advertisements in the national press, applicants

responded by standardized application form. Applicants

were pre-screened on the basis of university qualifications,

expected gradings, outside interests and positions ofresponsibility, and responses to open-ended questions as

to their reasons for applying. Short-listed applicants

subsequently underwent a 30-min structured interview by

telephone. Successful applicants following the structured

interview were invited to the 2-day assessment center.

Assessment Center

The assessment center took place over 2 days and involved

an intensive process of multi-rater observations of three

exercises targeting nine dimensions. Table 1 sets out the AC

exercise-dimension matrix in overview. In addition to thethree exercises (Analysis Report, Group Task, and Pre-

sentation), candidates gave verbal and numerical ability

tests and also completed an unstructured interview with

three assessors toward the end of the entire AC. The nine

dimensions evaluated were Analysis, Communication,

Planning and Organizing, Influencing Others, Team Work,

Leadership, Innovationand Change, Decision Making, and

Personal Effectiveness.

Procedure

A three-measurement timepoint, longitudinal design wasused. Time 1 questionnaires were administered immedi-

ately before the start of the AC while candidates were being

welcomed and briefed. Time 2 questionnaires were

completed immediately after the AC had finished. Note,

importantly, that outcome decisions from the AC were not

yet known by participants, this timing being essential to

remove any potential biasing effects of outcome decisions

upon their immediate reactions. Time 3 measures were

taken for both successful and unsuccessful candidates 6

months after the completion of the AC and once outcome

decisions were known by all participants. Successful

candidates were followed up at this stage within the

organization as all had commenced employment by thisstage. All 28 candidates to whom offers were made

accepted employment as this was a highly attractive

managerial training scheme. For rejected candidates, the

second author followed up responses using a postal

questionnaire to their home addresses.

Measures

In addition to biographical variables measured at Time 1

only (age, qualifications, ethnic minority status, and so

forth), measures were completed at all three timepoints

by participants. The questionnaire contained scales evalu-ating self-esteem, psychological well-being (the General

Health Questionnaire (GHQ) short-form version), and

positive and negative affect. Additionally, feedback

perceptions were evaluated only at Time 3 after the

organization had been able to give applicants formal

feedback on their performance, and career exploration

behavior was measured for unsuccessful and successful

applicants at Time 1 and Time 3 only. The details of

measures are as follows:

Self-Esteem. Rosenbergs (1989) standarized scale ofself-esteem was used. For the present sample, internal

consistencies (Cronbachs a) ranged from .77.88, and

testretest reliability was .82 (Time 1 to Time 2) and .85(Time 2 to Time 3).

Psychological Well-Being. The short-form versionGHQ (GHQ-12) was used to evaluate psyhological health

(Goldberg, 1992). Designed to detect non-psychotic

disorders in normal adult populations, this measure has

been widely used as a valid and reliable scale for measuring

key aspects of psychological health (Goldberg, 1992). The

present sample as ranged from .82 to .90; retest reliabilities

were above .70 between both measurement timepoints.

Note that as a measure of clinical disorder, higher scores on

the GHQ were originally scaled to indicate a higher

probability of disorder and thus indicate lower levels ofpsychological well-being. Scores were reverse coded in the

present study in order to simplify interpretation of

subsequent analyses such that higher scores indicate more

positive psychological well-being.

Positive and Negative Affect. The Positive and Nega-tive Affect Schedule (PANAS) was used (Watson, Clark, &

Tellegen, 1988). Consisting of 20 adjectives, this brief

schedule measures the two dimensions of positive and

negative affect. Ten adjectives describe negative moods

(distressed, upset, guilty, scared, hostile, irritable, ashamed,

Table 1. The assessment center matrix

Dimensions

Exercises

Analysis

report

Group

task Presentation

Analysis * *Communication * *

Planning and organizing * *

Influencing others * *

Team work *

Leadership *

Innovation and change * *

Decision making * *

Personal effectiveness * * *

Note: *Rated dimension by exercise.





6/21

nervous, jittery, afraid), while 10 adjectives describe

positive moods (interested, excited, strong, enthusiastic,

proud, alert, inspired, determined, attentive, active).

Reliabilities (as) for positive affect (PA) and negative

affect (NA) were a5 .89 and .85, respectively. The measure

was sensitive to changes over time in the current

study: testretest for PA was only .68 (Time 1 to Time 2)

and .54 (Time 2 to Time 3), and for NA this was only .71

(Time 1 to Time 2) and .45 (Time 2 to Time 3), indicating

that there were changes in respondents self-rated affect

over time.

Feedback Perceptions. Feedback perceptions were

measured with an eight-item scale derived from Dodd(1977) at Time 3 only. Cronbachs a was .88.

Career Exploration Behavior. The Career ExplorationSurvey (CES) by Stumpf, Colarelli, and Hartman (1983)

was used. Five of the CES scales were used: (1) environ-

mental exploration (a5 .83, e.g., I went to various career

orientation programmes), (2) self-exploration (a5 .88,

e.g., I have been retrospective in thinking about my

career), (3) intended/systematic exploration (a5 .74, e.g.,

I tried specific work roles to see if I like them), (4)

amount of information obtained (a5 .79, e.g., I currently

have a moderate amount of information on how Ill fit into

various career paths), and (5) Focus (a5

.86, e.g., Howsure are you that you know the type of organization you

want to work for?).

Results

Pre-Analysis Checks

Initial comparisons of biographical variables between the

accepted and rejected candidate groups were carried out.

Independent sample t-tests at Time 1 revealed no sig-

nificant differences on either gender (t51.34, p5NS), or

marital status (t5 .36, p5NS), but a significant

difference favoring younger applicants in age (t5 3.46,

po.001). Comparisons on other biographical variables

(number of assessment centers previously attended, educa-

tion level, and degree type) revealed non-significant

differences between the accepted and rejected applicants,

indicating that these sub-groups were essentially equivalent

and were not prone to underlying differences in composi-

tion, which could have confounded differences found on

study measures of mental health and career exploration

behavior.

Descriptive Statistics and Intercorrelations

Table 2 reports descriptive statistics, intercorrelations, and

reliability coefficients for all study variables at Time 1.

Cronbachs as are all acceptable at above .80. Overviewing

this correlation matrix, note that self-esteem correlates

positively and significantly with psychological well-being

(r5.20, po.05) and with positive affect (r5.48, po.01),

but negatively and significantly with negative affect

(r5 .22, po.05). The highest correlations were found

between positive and negative affect (r5 .66, po.01),

and interestingly between positive affect and careerexploration behavior (r5.54, po.01).

Longitudinal Differences in Accepted Vs. RejectedCandidates

Table 3 reports means, standard deviations, t-test values,

and effect size statistics for the key outcome variables of

mental health and career exploration behavior. Cohens d

(Cohen, 1988) was used to examine sub-group differences

on these variables between the rejected (i.e., majority) and

Table 2. Descriptive statistics and intercorrelations for Time 1 study variables

Mean SD

Intercorrelations

1 2 3 4 5 6 7 8 9

1. Age 23.02 2.46

2. Gender 1.50 .50 .00

3. Marital status 1.05 .22 .40** .04

4. Self-esteem 16.94 3.64 .04 .10

.13 .885. Psychological well-being: GHQ-12 10.71 4.64 .10 .16 .10 .20* .84

6. Positive affect 36.15 5.10 .09 .04 .14 .48** .32** .89

7. Negative affect 17.12 4.00 .02 .11 .02 .22* .32** .66** .85

8. Career exploration behavior 77.13 12.99 .13 .03 .10 .16 .02 .54** .02 .82

9. AC outcome decision 1.74 .44 .23* .13 .03 .11 .18 .10 .09 .18

Notes: Values on the diagonal are Cronbachs a coefficients.

GHQ, General Health Questionnaire.*po.05,**po.01, two-tailed tests.




7/21

Ta

ble3

.Su

b-g

roup

differenc

es

inmen

talhea

lthover

time:

accep

tedvs

.re

jec

tedapp

lican

ts

Time1

Time2

Time3

Time1

Time3

Accepted

Rejected

Accepted

Rejected

A

ccepted

Rejected

t

SignificanceSEd

SD

ratio

d

t

SignificanceSEd

SD

ratio

d

Mean

SD

Mean

SD

Mean

SD

Mean

SD

Mean

SD

mean

SD

Self-esteem

16.2

8

3.2

5

17.18

3.7

6

16.3

2

3.8

7

16.4

1

3.9

1

16

.07

3.7

5

16.7

3

3.9

9

1.1

9

NS

.22

.86

.2

5

.7

8

NS

.22

.94

.1

7

Psychological

well-being

12.0

7

5.1

2

10.19

4.3

7

11.8

1

4.8

7

10.4

2

3.8

8

10

.86

6.7

8

10.4

5

5.6

3

1.7

2

NS

.22

1.1

7

1.4

1

.28

NS

.22

1.2

0

.0

7

Positive

affect

37.1

8

3.8

5

35.88

6.2

0

35.4

3

5.8

5

34.3

6

7.9

1

33

.39

7.2

0

32.3

0

7.9

5

1.2

8

NS

.22

.62

1.2

5

.67

NS

.22

.91

.1

4

Negative

affect

16.8

6

4.0

0

17.77

4.6

0

12.8

2

3.5

4

13.0

0

3.5

9

13

.89

5.1

7

14.8

8

6.0

7

.9

9

NS

.22

.87

.2

3

.8

3

NS

.22

.85

.1

7

Career

exploration

behavior

81.0

0

11.4

3

75.73

13.3

0

80.9

0

12.2

6

77.8

4

13.2

1

62

.17

12.8

6

68.2

4

15.6

0

2.0

0

N

S1

(p5

.051)

.22

.86

1.4

1

1.8

7

NS

.22

.82

.4

3

Notes:d5

differencebetween

rejectedandacceptedmeansinstandarddeviationunits(effectssize).d

valuescomputedbyexpressingthedifferencesbetweenthe

meansoftherejectedandacceptedgroupsinpooledstandarddeviationunits.d5

(meanfortherejecte

dgroupmeanfortheacceptedgro

up)/SDpooled.

Positived

valuesindicateacceptedapplicantsscorehigher,negativethatrejectedapplicantsscorehigher.SEd5

standarderrorofd(unequalN

valuesforgroups).

SDratio

of

rejectedgrouptoacceptedgroupstandarddeviations,valuesgreaterthan1.0

0indicatealargerrejectedgroupstandarddeviation.

p5

.051forthist-valueoft5

2.0

0,

indicatingthattheTime1d

ifferencebetweenacceptedandrejectedgroupswasmarginallynon-significant.





8/21

accepted (i.e. minority) groups. According to Cohen, effect

sizes can theoretically range between positive and negative

infinity, although in personnel selection contexts it is rare

to find such extreme values (e.g., Anderson, Lievens, van

Dam, & Born, 2006; Ones & Anderson, 2002). As d in

effect indexes the standardized mean difference between

any two groups, it is particularly valuable when examining

differnces between sub-groups of varying sample sizes. Past

research into ethnic minority and gender differences inselection has made extensive use of such effect size

computations; here, we use Cohens d to compare

immediate and longer-term differences in psychological

well-being and career exploration behavior. Note that

according to Cohen (1988), effect sizesof zero to .20 can be

considered small, those around .50 are medium, and above

.80 are large in magnitude.

First, we consider differences in key variables at Time 1.

Two effect sizes approaching .50 were found. Accepted

candidates reported a higher base-line mean on psycholo-

gical well-being than that of subsequently rejected

candidates (means were 12.07 and 10.19, respectively,d5 .41). Subsequently accepted candidates also self-rated

more highly on career exploration behavior than

rejected applicants, suggesting that they were already more

actively involved in their job search (the means were 81.00

and 75.73, respectively, d5 .41). No other effect size

differences or t-test differences are evident in Table 3

between these two groups on base-line measures taken at

Time 1.

Second, differences between accepted and rejected

candidates are examined at Time 3, that is, 6 months

following this AC, and several interesting findings are

evident. Most importantly, no statistically significant

differences (according to t-test values) were found betweenaccepted and rejected candidates on self-esteem, psycholo-

gical well-being, positive or negative effect, or on the CES.

However, an effect size difference of d5 .43 was found

between accepted and rejected candidates on career

exploration behavior. Not surprisingly, this difference

was substantially higher for rejected applicants, perhaps

reflecting on-going job search behaviors for those still

active in applying for job vacancies elsewhere. A finding of

note here is that rejected applicants actually report a higher

mean (although also marginally higher SD value) on self-

esteem at Time 3 than accepted candidates (16.73 and

16.07, respectively), although this difference is non-significant (t5 .78, p5NS; d5 .17).

Third, receipt of a positive outcome decision did not

appear to improve values on self-esteem, positive affect, or

psychological well-being for accepted candidates. Time 1

to Time 3 mean ratings on self-esteem were 16.28

(SD53.25) and 16.07 (SD53.99), respectively; on

positive, affect values were 37.18 (SD53.85) and 33.39

(SD57.20); and on psychological well-being, 12.07

(SD55.12) and 10.86 (SD56.78), respectively. None of

these differences were significant, suggesting that there was

no longer-term PPEs from receipt of a successful select

decision at this AC.

Feedback Perception Differences

A final set of analyses were computed in order to examine

differences in feedback perceptions between the accepted

and rejected candidate groups. These are summarized in

Table 4. Large effect size differences in feedback percep-tions between the two groups are evident. For the eight-

item scale combined, d5 .86 (t54.26,po.001) on overall

feedback perceptions. Yet, the largest effect size difference

was found on the item Feedback information was

accurate: d51.08 (t56.52, po.001), strongly suggest-

ing that rejected candidates attributed unfavorable out-

come decisions to an inaccurate selection methodology.

Two other large effect size differences were found in

accordance with Cohens (1988) recommendations:

d5 .82 on Reference to good performance was made,

and d5 .93 on Feedback had an encouraging effect.

Note that feedback perceptions were also measured atTime 3 along with all other mental health scales and the

CES; for the former, we found several large effect size

differences between accepted and rejected candidates

(Table 4), but for the latter we found no sizeable or

statistically significant differences (Table 3).

Discussion

We believe that this paper makes two novel contributions

to the existing scant research into, and thus our under-

standing of, the psychological impact of assessment centers

upon applicants. First, we develop and expound anoperational definition of NPEs for all selection methods,

noting several facets of this definition and linking it to

recent research that has adopted the applicants perspective

to selection predictors. Second, although a relatively small-

scale primary study, we present the findings of a three-

timepoint longitudinal study into the NPEs and potential

PPEs of an operational assessment center in the United

Kingdom. Contrary to the findings of the Fletcher (1991)

study, which used identical measurement intervals to the

present investigation, we failed to find NPEs resulting from

this AC, most notably even though the effect size

differences between accepted and rejected candidates werelarge in reactions-level measures of applicant feedback

perceptions following this assessment center.

Non-Significant NPE Findings

Despite the fact that 6 months after the AC, and indeed in

most cases some months after receiving feedback on their

performance, rejected candidates reacted far more nega-

tively to the feedback reactions measure than did accepted

candidates (see Table 4), the overall feedback perception




9/21

effect size difference across all eight items combined was

large (d5

.86). It appears that rejected applicants attrib-uted this negative decision to a lack of accuracy in the

assessment process (d51.08, t56.52, po.001) in spite of

this being a 2-day, job-relevant assessment center. Detailed

feedback was made available to all candidates, regardless

of the outcome of the AC final decision. Again, it seems

that the wisdom of providing all applicants, especially

those who have been unsuccessful, with highly detailed

feedback is more questionable than was previously thought

by selection psychologists (Schinkel et al., 2004).

Undoubtedly, the key findings of this study are that in

contrast to these highly significant findings of feedback

effect differences, rejected applicants suffered no serious

declines in self-esteem, psychological well-being, positiveaffect, or indeed expounded career exploration behavior as

a result of being unsuccessful at this assessment center. This

is noteworthy in that this AC was for a highly prestigeous

organization that is perceived by applicants as being a

highly attractive employer. One explanation here is that the

selection ratios involved in being successful for this

organization were so competitive for applicants that their

perceptions of being unsuccessful could have been moder-

ated by this selection ratio. Various perceptual, psycholo-

gical, and attributional biases may therefore have allowed

unsuccessful candidates in this particular situation to

persuade themselves that in effect so many otherapplicants were also unsuccessful that I knew all along

that this was a long-shot. Counter to this explanation is

the fact that this was the final stage of the selection process

for the organization, that is, that applicants had success-

fully survived several stages of shortlisting and assessment

previously and this AC represented the final barrier to entry

to employment. Despite the job relevance of exercises at

this AC, trained assessors, and detailed feedback including

behavioral exemplars being given to rejected candidates,

they still construed to attribute this to inherent inaccuracies

and unjust elements in the method itself (see also Fletcher

& Kerslake, 1992).That unsuccessful applicants suffered no longer-term (or

at least after a 6-month period) NPEs or deleterious

impacts upon the career exploration behavior suggests

robust psychological barriers and perceptual screening

heuristics between their immediate-level reactions and

longer-term psychological impacts. This is an important

finding, although of course this is only a single study of a

single selection predictor method. It could also have been

the case that many of these rejected applicants had found

employment elsewhere during this period; unfortunately,

our follow-up measure did not include items to explore the

present employment status of those individuals. Further

research is called for to examine this issue. Yet, as arguedearlier in this paper, the AC is precisely the method that

might be expected to cause NPEs in rejected candidates. It

is noteworthy therefore that this study failed to find any

evidence of longer-term negative effects, and this strongly

suggests that future research in this area needs to

distinguish clearly between differences that are reactions-

level preference responses and longitudinal effects upon

applicant decision-making, psychological well-being, and

even career exploration behavior (Noe & Steffy, 1987).

Non-Significant PPE Findings

One further finding of relevance here is that we likewise

found no strong PPEs from being successful at this AC.

Note that self-esteem, psychological well-being, and

positive affect did not improve markedly either at Time 2

measures, that is, before the actual decision was known but

immediately following the AC procedure, or 6 months later

at Time 3. Rather, we found some evidence of declines in

psychological well-being for successful applicants 6

months after the AC and in most cases some months into

their job tenure. During the intervening period, most of

Table 4. Differences in feedback perceptions: accepted vs. rejected sub-groups

Feedback perception

Accepted Rejected

t-value SEd SDratio dMean (SD) Mean (SD)

Feedback provided 2.89 (.79) 2.28 (1.04) 3.18** .22 .76 .60

Reference to good performance 2.96 (.94) 2.11 (.99) 3.99*** .23 .95 .82

Reference to poor performance 2.57 (1.03) 2.24 (1.04) 1.43* .22 .99 .31

Feedback had an encouraging effect 3.03 (1.26) 1.88 (1.09) 4.25*** .23 1.16 .93Understanding of feedback information 3.29 (.90) 2.68 (1.15) 2.81** .22 .78 .54

Feedback information was accurate 3.59 (.75) 2.34 (1.10) 6.52*** .23 .68 1.08

Valuable information for personal development 3.21 (1.17) 2.16 (1.37) 3.87*** .23 .85 .76

Provision of developmental recommendations 2.07 (1.33) 2.07 (1.13) .01* .22 1.18 .00

Overall feedback perceptions scale 23.69 (5.81) 17.79 (6.58) 4.62*** .23 .88 .86

Notes: SEd5 standard error of d.*po.05,**po.01,***po.001.





10/21

these applicants had already joined the organization and it

could have been that widely documented socialization

effects had eliminated any short-term increases in well-

being and core self-esteem (e.g., Saks, 1995; Saks &

Ashforth, 1997). Again, caution is warranted over the

interpretation of these findings as the sample size in the

present study is relatively small, so these findings should be

tempered with some caution. Future research is certainly

called for to examine both NPE and PPE effects inassessment centers and other selection methods; here, it is

appropriate to highlight that different time periods of

measurement also need to be included in future studies in

order to test the possibility that effects may be present but

might only remain manifest for some weeks rather than

months after receipt of the outcome decision.

Brief Comments over Future Directions for NPE andPPE Research

The present investigation contributes to the rather limitednumber of studies into the possible psychological effects,

either negative or positive, of selection methods upon

applicants. As noted earlier in this paper, it is surprising

that this area has attracted so little attention by psychol-

ogists active in employee selection research. As a con-

sequence, several major avenues for future research,

theoretical model-building, and empirical study are im-

mediately apparent. In the opening section of this paper,

two working definitions of NPEs and PPEs were postu-

lated; it is likely that other researchers will have rather

different opinions of these constructs and so may wish to

put forward somewhat differing formulations of these in

due course. Additional research would, it would at leastseem, most valuable to commence the mammoth task of

quantifying the direct effects and costs of NPEs upon

reduced job performance; that is, to investigate the utility-

related costs of NPEs to organizations who may well be

unaware of this possibility. Investigation of how such costs

can be moderated by the provision of accurate but tactful

feedback to unsuccessful internal candidates would be a

follow-up line for initial research. In addition to both of

these performance- and utility-related directions for future

studies, however, we would suggest that research into both

NPEs and PPEs as a psychologically related strategy is fully

warranted without the need to justify this theme of researchon what are essentially economic grounds. It is beholden

upon selection psychologists to investigate and quantify the

psychological impacts of the assessment methods used in

employee recruitment and selection settings, especially

where these may result in NPEs and outcomes. Further

research is clearly called for in this regard not just into the

NPEs of assessment centers but on other popular predictors

such as structured interviews, tests of general mental ability

and personality, job sample tests, and situational judgment

inventories, among others.

Conclusion

In comparison even with the growing body of research into

immediate-level applicant reactions, the potential for

predictor methods to provoke NPEs upon applicants has

remained a critically underresearched topic. Curiously, this

dark side of the moon, as Anderson (2004) refers to it,

has received notably little research attention by selection

psychologists despite health psychology and research intonegative psychological impacts of workload being vibrant

areas of study within IWO psychology more generally. This

paper represents an initial investigation into a sub-field that

first requires definitional clarity and conceptual distinction

from existing research efforts into immediate-level reac-

tions outcomes. To conclude, this study, and its principal

finding of a lack of longer-term NPEs upon applicants

caused by a final-stage assessment center, can only be cited

as one of the initial steps toward establishing this important

field of enquiry for further empirical investigation.

Acknowledgement

We wish to express our thanks to Sonja Schinkel for her

comments and suggestions on an earlier version of this

paper.

References

American Psychiatric Association (1994) Diagnostic and statisticalmanual of mental disorders, 4th Edn (DSM-IV). Washington,DC: APA.

Anderson, N. (2004) Editorial the dark side of the moon:

Applicant perspectives, negative psychological effects (NPEs),and candidate decision making in selection. International

Journal of Selection and Assessment, 12, 18.Anderson,N. (2005) Relationships between practice and research in

personnel selection: Does the left hand know what the right isdoing? In A. Evers, N. Anderson and O. Smit-Voskuyl (Eds),Handbook of selection (pp. 124). Oxford: Blackwell.

Anderson, N., Born, M. and Cunningham-Snell, N. (2001)Recruitment and selection: Applicant perspectives and out-comes. In N. Anderson, D.S. Ones, H. Sinangil and C.Viswesvaran (Eds), Handbook of industrial, work and organiza-tional psychology, Vol. 1: Personnel psychology (pp. 200218).London: Sage Publications Ltd.

Anderson, N., Lievens, F., van Dam, K. and Born, M. (2006) Aconstruct-driven investigation of gender differences in a leader-

ship-role assessment center. Journal of Applied Psychology, 9,219240.

Bauer, T.N., Maertz, C.P., Dolen, M.R. and Campion, M.A. (1998)Longitudinal assessment of applicant reactions to employmenttesting and test outcome feedback. Journal of Applied Psychol-ogy, 83, 892903.

Bauer, T.N., Truxillo, D.M, Paronto, M.E., Campion, M.A. andWeekley, J.A. (2004) Applicant reactions to different selectiontechnology: Face-to-face, interactive voice response, and com-puter-assisted telephone screening interviews. International

Journal of Selection and Assessment, 12(2), 135148.Chan, D. and Schmitt, N. (2004) An agenda for future research on

applicant reactions to selection procedures: A construct-oriented




11/21

approach. International Journal of Selection and Assessment,12, 923.

Cohen, J. (1988) Statistical power analysis for the behavioralsciences, 2nd Edn. Mahwah, NJ: Erlbaum.

Derous, E., Born, M. and De Witte, K. (2004) How applicants wantand expect to be treated: Applicants selection treatment beliefs

and the development of the social process questionnaire onselection. International Journal of Selection and Assessment, 12,99119.

Dodd, W.E. (1977) Attitudes toward assessment centre programs.

In J.L. Moses and W.C. Byham (Eds), Applying the assess-ment centre method (pp. 132146). Elmsford, NY: PergamonPress.

Fletcher, C. (1991) Candidates reactions to assessment centres andtheir outcomes: A longitudinal study. Journal of OccupationalPsychology, 64, 117127.

Fletcher, C. and Kerslake, C. (1992) The impact of assessment

centres and their outcomes on participants self-assessments.Human Relations, 45, 281289.

Gilliland, S.W. (1994) Effects of procedural and distributive justiceon reactions to a selection system. Journal of AppliedPsychology, 79, 117127.

Gilliland, S.W. and Steiner, D.D. (2001) Causes and consequencesof applicant perceptions of unfairness. In R. Cropanzano

(Ed.), Justice in the workplace (pp. 175195). Hillsdale, NJ:Erlbaum.

Goldberg, D. (1992) General health questionnaire (GHQ-12).Windsor, UK: NFER NELSON.

Hart, P.M. and Cooper, C.L. (2002) Occupational stress: Toward amore integrated framework. In N. Anderson, D.S. Ones, H.

Sinangil and C. Viswesvaran (Eds), Handbook of industrial andorganizational psychology, Vol. 2 (pp. 93114). Thousand Oaks,CA: Sage Publications.

Hausknecht, J.P., Day, D.V. and Thomas, S.C. (2004) Applicant

reactions to selection procedures: An updated model and meta-analysis. Personnel Psychology, 57, 639683.

Horvath, M., Ryan, A.M. and Stierwalt, S.L. (2000) The influenceof explanations for selection test use, outcome favorability, and

self-efficacy on test-taker perceptions. Organizational Behavior

and Human Decision Processes, 83, 310330.Iles, P. and Robertson, I. (1997) The impact of personnel selection

procedures on candidates. In N. Anderson and P. Herriot(Eds),International handbook of selection and assessment (pp. 543566). Chichester, UK: Wiley.

Imus, A.L. and Ryan, A.M. (2005) Relevance and rigor in research

on the applicants perspective: In persuit of pragmatic science. InA. Evers, N. Anderson and O. Voskuijl (Eds), Handbook ofSelection (pp. 280296). Oxford, UK: Blackwell.

Judge, T.A. and Bono, J.E. (2001) Relationship of core self-

evaluation traits self-esteem, generalized self-efficacy, locusof control, andemotional stability with jobsatisfaction andjobperformance: A meta-analysis. Journal of Applied Psychology,86, 8092.

Lievens, F. and Klimoski, R.J. (2001) Understanding the assessmentcenter process. Where are we now? In C.L. Cooper and I.T.Robertson (Eds), International review of industrial and organi-sational psychology, Vol. 16 (pp.245286). Chicester, UK: JohnWiley & Sons.

Lievens, F., De Corte, W. and Brysse, K. (2003) Applicantperceptions of selection procedures: The role of selection

information, belief in tests, and comparative anxiety. Interna-tional Journal of Selection and Assessment, 11, 6777.

Lounsbury, J.W., Bobrow, W. and Jensen, J.B. (1989) Attitudestoward employment testing: Scale development, correlates and

known-group validation. Professional Psychology: Researchand Practice, 20, 340349.

Macan, T.H., Avedon, M.J., Paese, M. and Smith, D.E. (1994) The

effects of applicants reactions to cognitive ability tests and an

assessment centre. Personnel Psychology, 47, 715738.Noe, R.A. and Steffy, B.D. (1987) The influence of individual

characteristics and assessment centre evaluation on career

exploration behavior and job involvement. Journal of Voca-

tional Behavior, 30, 187202.Ones, D.S. and Anderson, N. (2002) Gender and ethnic group

differences on personality scales in selection: Some British data.

Journal of Occupational and Organizational Psychology, 75,255276.

Ployhart, R.E. and Harold, C.M. (2004) The applicant attribution-

reaction theory (AART): An integrative theory of applicant

attributional processing. International Journal of Selection and

Assessment, 12, 8498.Ployhart, R.E. and Ryan, A.M. (1997) Toward an explanation of

applicant reactions: An examination of organizational justice

and attribution frameworks. Organizational Behavior and

Human Decision Processes, 72, 308335.Ployhart, R.E. and Ryan, A.M. (1998) Applicants reactions to the

fairness of selection procedures: The effects of positive rule

violations and time of measurement. Journal of Applied

Psychology, 83, 316.Ployhart, R.E., Ryan, A.M. and Bennett, M. (1999) Explanations

for selection decisions: Applicants reactions to informational

and sensitivity features of explanations. Journal of Applied

Psychology, 84, 87106.Robertson, I.T., Iles, P.A., Gratton, L. and Sharpley, D. (1991) The

impact of personnel selection and assessment methods on

candidates. Human Relations, 44, 963982.Rosenberg, M. (1989) The self-esteem scale. Windsor, UK: NFER

NELSON.Ryan, A.M. and Ployhart, R.E. (2000) Applicants perceptions of

selection procedures and decisions: A critical review and agenda

for the future. Journal of Management, 26, 565606.Ryan, A.M., Greguraas, G.J. and Ployhart, R.E. (1996) Perceived

job relatedness of physical ability testing for firefighters:

Exploring variations in reactions. Human Performance, 9,

219240.Saks, A.M. (1995) Longitudinal field investigations of the moderat-

ing and mediating effects of self-efficacy on the relationship

between training and newcomer adjustment. Journal of AppliedPsychology, 80, 211225.

Saks, A.M. and Ashforth, B.E. (1997) Organization socialization:

Makingsense of thepast andpresent as a prologue forthe future.

Journal of Vocational Behavior, 51, 234279.Salgado, J.F. (1999) Personnel selection methods. In C.L. Cooper

and I.T. Robertson (Eds), International review of industrial andorganisational psychology (pp. 154). Chichester, UK: JohnWiley & Sons.

Salgado, J.F., Viswesvaran, C. and Ones,D.S. (2002) Predictors used

for personnel selection: An overview of constructs, methods

and techniques. In N. Anderson, D.S. Ones, H. Sinangil and

C. Viswesvaran (Eds), Handbook of industrial, work and

organizational psychology, Vol. 1. London: Sage.Schinkel, S., van Dierendonck, D. and Anderson, N. (2004)

The impact of selection encounters on applicants: An experi-

mental study into feedback effects after a negative selection

decision. International Journal of Selection and Assessment, 12,197205.

Schmit, M.J. and Ryan, A.M. (1992) Test-taking dispositions:

A missing link? Journal of Applied Psychology, 77, 629637.Schmit, M.J. and Ryan, A.M. (1997) Applicant withdrawal: The

role of test-taking attitudes and racial differences. Personnel

Psychology, 50, 855876.





12/21

Schmitt, N. and Chan, D. (1999)The statusof research on applicant

reactions to selection tests and its implications for managers.

International Journal of Management Research, 4561.Schmitt, N., Oswald, F.L., Kim, B.H., Gillespie, M.A. and Ramsay,

L.J. (2004) The impact of justice and self-serving bias explana-

tion of the perceived fairness of different types of selection tests.

International Journal of Selection and Assessment, 12, 160171.Schuler, H. and Fruhner, R. (1993) Effects of assessment center

participation on self-esteem and on evaluation of the selection

situation. In H. Schuler, C.J.L. Farr and M. Smith (Eds),Personnel selection and assessment: Individual and organiza-

tional perspectives (pp. 109124). Mahwah, NJ: Lawrence

Erlbaum.

Smither, J.W., Reilly, R.R., Millsap, R.E., Pearlman, K. and Stoffey,R.W. (1993) Applicant reactions to selection procedures.Personnel Psychology, 46, 4976.

Stumpf, S.A., Colarelli, S.M. and Hartman, K. (1983) Developmentof the career exploration survey (CES). Journal of VocationalBehaviour, 22, 191226.

van Vianen, A., Taris, R., Scholten, E. and Schinkel, S. (2004)Perceived fairness in personnel selection: Determinants andoutcomes in different stages of the assessment procedure.

International Journal of Selection and Assessment, 12, 149159.Watson, D., Clark, L.A. and Tellegen, A. (1988) Development andvalidation of brief measures of positive and negative affect: ThePANAS scales. Journal of Personality and Social Psychology, 5,10631070.




13/21

Ta

bleA1

.Chrono

log

ica

lsum

maryo

fresearc

hintopsyc

ho

log

ica

le

ffec

tso

fse

lec

tionme

tho

dsupon

app

lican

ts

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

Lounsburyetal.

(1989)

Attitudestoward

employmentte

sting

n5

546for

telephoneinterview

n5

112applicants

formanualwork

position

Testingingene

ral

Opinionsof

employmenttesting

Study1:

Study2:post-testor

post-feedbacktest

Sex,age,race,

occupation,

education

attainment,

experiencewith

testing

Attitudestotes

t

takingperceive

d

fairnessoftests

Instudy2Hispanics

andyounger

applicantshadmore

positiveattitudes

Significantlymore

favorableattitudes

wereobserved

towardemployment

testingwhenjob-

relationhadbeen

explainedand

feedbackbeengiven

thanifithadnot

Rejectedapplicants

hadmorenegative

attitudesthan

selectedorhold-

fileapplicants

Robertsonetal.

(1991)

Beliefsaboutthe

adequacyofse

lection

procedures,

perceivedcareer

impact

n5

170

Employeesofa

financialinstitution

Biodata,situational

interview,

assessmentcenter

Withdrawal

cognitions

Organizational

commitment

Psychologicalhealth

Self-esteem

Atspecificpoin

ts

duringthe

assessmentprocess

Select/reject

Withdrawal

cognitions

Organizational

commitment

Psychological

health

Perceivedcareer

impactwas

significantlygreater

forrejectedthanfor

acceptedcandidates

Acceptedand

rejectedcandidates

didnotdifferon

psychologicalhealth

orself-esteem

Candidatesinthe

assessmentcenter

phaseweremore

concernedwith

adequacythan

candidatesinthe

situationalinterview

phase

Acceptedcandidates

exhibitedhigher

organizational

commitmentand

believedmethods

AppendixA





14/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

weremoreadequate

thanrejected

candidates

Fletcher(1991)

Psychological

well-being

Needforachievement

Organizational

commitment

Jobinvolvement

n5

57

Highergradeclerical

staff

Assessmentce

nter

(groupdiscussion,

in-tray,objective

setting,psycho

metric

tests,self-repo

rt,

interview)

Self-esteematwork

anddepressedmood

atwork

Pre-test

Post-test(straight

afterassessme

nt)

Post-test(6months

later)

Select/reject,

assessmentcenter

participation

Psychological

well-being

Successful

candidateshad

significantlylower

scoresthan

unsuccessful

candidateson

depressedmoodat

work

Self-esteemincreases

fromT1toT2and

dropsatT3toalevel

lowerthanatT1

Unsuccessful

candidatesshoweda

significantlylower

levelofself-esteemat

T3

Smitheretal.(1993)Perceivedpred

ictive

validityandface

validity,job

relatedness,

proceduraland

distributivejus

tice

n15

154entry-level

andrecruiting/

employment

managers

n25

1895

applicantsforawide

rangeofcivilservice

jobs

Descriptionsofeight

cognitivetestsandsix

otherprocedures

(personality

inventory,

in-basket,

leaderlessgrou

p

discussion,

bio

data,

unstructured

interview,and

structuredinterview)

Jobrelatedness

(predictivevalidity

andfacevalidity)

organizational

attractiveness,

likelihoodof

improvement,

perceivedknowledge

ofresults,affect,

cognitiveability

Study1:post-test

Study2:post-test

andpost-feedb

ack

testafter1mo

nth

Study1:perceived

predictivevalidity,

facevalidityGPA,

andcognitivetest

performance

Study2:

organizational

attractiveness,

likelihoodof

improvement,

perceivedknowledge

ofresults,affect

Perceptionsofjob

relatedness,

proceduraljust

ice,

distributivejustice

andrecommen

dation

intentions

Perceptionsof

validitywerenot

enhancedthrough

facevalidity

manipulations

Perceivedvaliditydid

notinescapably

correspondwiththe

actualvalidityofthe

selectionprocedures

Applicantsreactions

werepositively

relatedto

organizational

attractiveness,

justice

perceptionsand

willingnessto

recommendthe

employertoothers

Higher-ability

applicantswerenot

morelikelyto

perceiveselection




15/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

proceduresaslessjob

related

Proceduraljusticeas

wellas

recommendation

intentionswere

positivelyrelatedto

affectandlikelihood

ofimprovementand

perceivedpredictive

validity

Gilliland(1994)

Proceduralfairness,

distributivefairness

n5

270

undergraduate

studentsparticipating

inexchangefor

financialrewards

(whenselected)and/

orcredits(when

rejected)

Worksampletest,

cognitiveabilitytest,

overtintegritytest

Fairnessperceptions

Recommendation

intentions:

Self-efficacy

Performance

Post-feedback

test

Hiringexpectation

Perceivedfairness,

recommendation

intentions,self-

efficacy,performance

Selectedindividuals

perceivedgreater

fairnessinthe

selectionprocessand

outcomedecision

thanrejected

individuals,andthis

effectwasmore

pronouncedwhen

individualshadhigh

expectationsofbeing

hired

Rejectedapplicants

weremorelikelyto

recommendwhen

givenexplanationsfor

rejectdecision

Fairprocedureslead

tohigherself-efficacy

forselected

candidatesandlower

forrejected

Ryan,

Greguras,and

Ployhart(1996)

Self-efficacy

Job-relatedness,

experience,

consistencyof

test

administration

N5

81Fire

Firefightersattending

aweekofsessionsat

astatefireschool

3.7

%female

81.4

8%male

Sevenphysicalability

tests(PAT)

NA

Self-efficacy,

experience,current

fitnesslevel,

departmentfitness

climate

Perceived

job-relatedness

Self-efficacyfora

specificPATwas

positivelyrelatedto

theperceivedjob-

relatednessofthat

PAT

Individualswith

higherself-efficacy

sawPATs





16/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

PloyhartandRyan

(1997)

Processandou

tcome

fairness

n5

80297

applicantsto

graduateschoolin

psychology

GRE-scores,GPA,

researchandw

ork

experiences,re

search

interests,lettersof

recommendation,

personalstatem

ent

Organizationaljustice

Processfairness

Outcomefairness

Selectiondecision

Perceivedcausefor

decision

Self-efficacy

Self-esteem,

Self-assessed

performance

Recommendation

intention

Applicationintention

Acceptanceintention

Reapplication

intention

Expectancyoffuture

success

Pre-test(before

application),post-

test

(afteroffer)

Admissionstatus,

attributionsfor

decision

Recommendation

Application

Acceptanceand

reapplication

intentions

Self-efficacy

Self-esteem

Self-assessed

performance

Morefairlyperceived

proceduresleadto

morefavorable

intentionsandself-

assessed

performance.

Effectincreasedas

outcomefairness

increased

Applicantswho

perceivedprocedures

tobeunfairandwere

selectedreported

lowerself-efficacy

Self-servingbiasonly

occurredwhen

individualsperceived

fairprocedures

SchmitandRyan

(1997)

Test-takingattitudes,

withdrawal

cognitions

n5

3290Applicants

forthepostofpolice

officer,ofwhom

n5

618were

interviewedafter

withdrawal

Surveysand

Interviews

Test-takingattitudes

Pre-test

Anxiety

Motivation

Race

Withdrawal

Higherlevelsof

anxietyand

motivationin

applicantsleadto

lowerlikelihoodof

withdrawal

Caucasians

possessedslightly

highertest-taking

motivation,wereless

anxious,andbelieved

moreintheefficacyof

thetestscomparedto

African-Americans

Test-takingattitudes

onlypossessasmall

relationtowithdrawal

PloyhartandRyan

(1998)

Processandou

tcome

fairness

n5

283

undergraduate

studentsparticipating

inexchangefor

credits

CognitiveabilitytestFairness,

Intentions,Job

acceptance,

Future

experiment,

Reapplication,

Recommendation

Pre-testpost-te

st

post-feedback

questionnaire

Select/reject,

Administrative

consistency

Performance

expectations,job

acceptance

intentions,futu

re

experiment

intentions,

Positive

inconsistency

generallydoesnot

resultindifferent

perceptionsof

fairnesscompared




17/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

Self-perceptions,

Withdrawal

reapplication

intentions,

recommendation

intentions,

withdrawal

withconsistency

Post-hirejob

acceptancewas

unaffectedbyprocess

fairness

Unfairlyperceived

proceduresleadto

negativeintentions.

Themostunfavorable

reactionsoccurred

withunfairly

perceivedprocedures

andafairoutcome

Pre-hiringintentions

andoutcomeare

centraltotheforming

ofreactionsbecause

oftheirinfluenceon

post-hireintentions

andfairness

PloyhartandRyan

(1998)

Processfairnes

s,

explanationadequacy

n15

156

undergraduate

studentsparticipated

inexchangefor

credits

n25

35entry-level

graduatestudents

justinformedonthe

decision

Study1:Scenarioof

cognitiveabilitytest,

andjobknowledge

teststaken

Study2:GRE,

GPA,

researchintere

st,

personalstatem

ent,

researchexperience,

workexperienc

e,

letterof

recommendation

Processfairness

Reportedself-

perceptions,

Organizational

perceptionsletter

Informationaland

sensitivity

explanation,select/

reject

Perceptionsof

processfairnesswere

enhancedby

providingpersonalor

procedural

information

Reportsofself-

perceptionsfor

rejectedparticipants

wereharmedby

providingpersonal

informationand

enhancedby

providingadiversity

justification.

Vice

versaforaccepted

applicants

Reportsonself-

perceptionswere

mostfavorablewhen

researchinterest,

workexperience,or

researchexperience

informationwas

provided





18/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(s)

Key

findings

Derousetal.(2004)Selectiontreatment

beliefs

n15

660senior-

yearstudentsat

campusrecruitment

event

n25

634applicants

fordifferentbranches

ofindustry

SocialProcess

Model

ofSelection(SPS

model)

Socialprocess

questionnaireon

selectionimportance

(SPQS-I),

Expectedrealization

(SPQS-E)!

only

theapplicantsfilled

outthisquestionnaire

Students:SPQS-I

questionnairea

fter

screeningprocedure

onthesameday

Applicants:

SPQS-IandSP

QS-E

afterinitialscreening

andbeforeactual

testing

Gender,workstatus,

educationallevel,

motivationtoapply,

priorselection

experience

Selectiontreatment

importance,

expectedselection

treatment

Supportforstability

ofsix-factormodel

(SPQS-I)

Workstatusdidnot

effectvaluationand

expectationof

treatment

Motivationandprior

selectionrelatedto

SPQS-Ebutnotto

SPQS-I

Educationrelatedto

SPQS-Ebutnotto

SPQS-I

Femalesvaluedall

treatmentfactors

higherthanmales(in

SPQS-I)

Bauer,Truxillo,

Paronto,

Weekley,

andCampion(2004)

Selectionprocedural

justice

n5

153students

Face-to-face

interviewscree

nings,

interactivevoic

e

response(IVR)

screenings,

computer-assis

ted

telephone-screening

interviews

(contentofscreening

isidenticalfor

3

screeningmeth

ods)

WonderlicPersonnel

test,

NEO(measures

conscientiousness),

selectionprocedural

justicescale(SPJS)

!

11dimensions

offairness

Time1:inclassroom,

questionnaires

to

measureindependent

variables

Time2:

IVRcondition:

studentshadtocall

forautomatic

screeningandtook

surveyonwebsite

TIcondition:

studentswerecalled

forscreeningandtook

surveyonwebsite

FTFcondition:

livescreening

interviewatcampus.

Afterscreening

taken

tocomputerforfinal

survey

Cognitiveability,

conscientiousness,

age,gender,work

experience

Processandou

tcome

fairness

IVRwasratedlower

oninterpersonal

treatment,two-way

communicationand

openness(IVRisnon-

interpersonal)

Nodifferenceson

otherprocedural

justicefactors

betweenIVRand

othermethodsof

screening

vanVianenetal.

(2004)

Fairnessinpersonnel

selection

n5

282applicants

fordifferentjobsin

different

organizations

Psychometrictests,

feedbackinterview

Testbeliefs,

perceivedfairnessof

theselection,

perceptionofthe

Time1:pre-testing

Time2:post-testing,

pre-feedback

Time3:post-

feedback

Opennessto

experiences,age,

education

Testbeliefs,

perceivedjob

relatedness,

perceived

performance,

Significanteffectof

previoustest

experiencesontest

beliefs

Opennessto




19/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(s)

Key

findings

feedback-interview,

jobattractiveness

pre-andpost-

feedbackfairne

ss

measures,perc

eived

feedbacktreatm

ent,

feedbackconte

nt,

jobattractivene

ss

experienceinfluenced

perceptionsof

performance,

feedbackandfairness

Perceived

performanceand

perceivedjob

relatednesswere

independentlyrelated

topre-feedback

fairnessperceptions

Post-feedback

fairnessperceptions

werepositively

affectedbypre-

feedbackfairness

perceptions,

feedbackcontentand

perceivedfeedback

treatment.Itwasnot

relatedtojob

attractiveness

Perceivedfeedback

treatmentand

feedbackcontent

directlyaffectedjob

attractiveness

Schmittetal.(2004)

n5

644freshmen

students

Situational

JudgementInventory

(SJI)

Biographicalitems,

situationaljudgment

items,reactionitems

onperformance

beliefs,

test

relevance,

fairness

Reactionitems

directlyafterth

e

bio-dataand

SJIitems

Biographicalitems

andsituational

judgmentitems

in12performance

dimensions

(measuresprevious

experience,

interest,

motivation),ACT/SAT

test

Performancebeliefs

(absoluteand

comparative),

perceptionsoftest

relevance,

fairness

Relevanceand

fairnessaredirectly

influencedby

performancebeliefs

(consistentwithself-

servingbias)

Indirecteffectof

performancebeliefs

onfairnessmediated

byrelevance

perceptions

(consistentwith

justiceperspective)

Relevanceand

performanceinteract

intheirimpacton

fairnesswhen

comparative





20/21

Ta

bleA1

.Con

tinue

d

Author(s)

Construc

ts

measure

d

Sample

Predicto

r

method

Measures

Measurem

ent

timing

Independent

variable(s)

Depende

nt

variable(

s)

Key

findings

performanceas

opposedtoabsolute

performanceis

considered

Minoritiesreactless

favorablyto

standardizedtests

thanmajorities

Schinkeletal.

(2004)

Feedbackeffec

t

n5

119students

GMAtests

Affectivewell-being

scal

Documents

Negative Psychological Effects of Selection Methods Construct Formulation and an Empirical Investigation Into an AC