Goal Diffulty (3)

8/14/2019 Goal Diffulty (3)

1/9

Journal of Applied Psychology1990, Vol. 75, No. 3, 227-234 Copyright 1990 by the American Psychological Association, Inc.OO21-9OIO/9O/ OO.75

Operationalization of Goal Difficulty as a Moderatorof the G oal Difficulty-Performance RelationshipPatrick M . WrightDepartment of ManagementUniversity of Notre D ame

Examined the research studies cumulated in recent quantitative reviews of the relationship be-tween goal difficulty and performance to determine how goal difficulty has been operationalized.4 categories (assigned goal level, self-set goal level, performance improvement, and difficultyperceptions) of operationalization were discovered, and th e operationalization ofgoal difficultywas tested as a moderator of the relationship between goal difficulty and performance. Strongsupport for this moderating role was found; the different operationalizations accounted for 26%ofthe variance in effect sizes. Implications for operationalizing goal difficulty in future goal settingresearch are discussed .

The effectiveness of goal setting as a motivational too l is sel-dom questioned. Substantial empirical evidencehassupportedthe proposition that specific, difficult goals lead to higher per-formance than do instruction s to do your best or easy goals(Locke, Shaw, Saari, & Latham, 1981). On the basis of theirrecent meta-analyses, both M ento, Steel, and K arren(1987)andTubbs (1986) concluded that the relationship between goal dif-ficulty and performan ce has been w ell established. In fact,Mento et al. (1987) stated, If there is ever to be a viable can di-date from the organizational sciences for elevation to the loftystatus of a scientific law of nature, then the relationships be-tween goal difficulty, difficulty/specificity and task perfo r-man ce are most worthy of serious consideration (p. 74).Conclusions such as this may need to be tempered becausequite different results were obtained in these two meta-ana-lyses, despite the fact that the two had a large percentage ofstudies in common . Tubbs(1986)found an effectsizeof .816 forgoal difficulty, whereas Mento et al. (1987) reported an effectsize of.581.In addition, Tubbs and Mento observed differentresults regarding moderators of the relationship between goaldifficulty and performance. Tubbs (1986) found that both thesetting (laboratory vs.field and the manner in which goal dif-ficulty was operationalized (direct measurem ent vs. subjective)mo derated the goal difficulty-performance relationship ,whereas Mento etal.(1987) found no consistent mod erators ofthis relationship. Wood, M ento, and Locke (1987) added three

studies to the Mento et al. (1987) meta-analysis, however, andfound that task complexity explained 5.7% of the variance ineffect sizes.

This research was partially supported by a grant from the Center forResearch in Business, University of Notre D ame.I would like to extend special thanks to John Hollenbeck, RobertVecchio, Howard K lein, the late Stephen Prem ack, and three anony-mous reviewers for their valuable commen ts on an earlier version ofthe manuscript. 1 also thank Robert Wood, Anthony Mento, andEdwin Locke for providing their task complexity ratings.Correspondence co ncerning this article should be addressed to Pat-rick M. Wright, whoisnow at the Dep artment of Management, Texas

A&M University, College of Business, College Station, Texas77843-4221.

In this study I examined th e possible moderating role of theoperationalization of goal difficulty in the relationship be-tween goal difficulty and performance. This examinationseems warranted for three reasons. First, Latham, Erez, andLocke (1988) found tha t the large variance in the relationshipbetween participation and performance was attributable tominor differences in how the participation manipulation wasoperationalized. Second, both Mento et al. (1987) and Tubbs(1986) found significant unexplained variance in effect sizesacross the goal difficulty-performan ce literature. Finally,Tubbs(1986)found some support forthemoderatingroleof theoperationalization of goal difficulty in the goal difficulty-performance relationship.Thus, this study tested the hypothesis that the operationali-zation of goal difficulty moderates the relationship betweengoal difficulty and performance. It goes beyond Tubbs's (1986)analysis of operationalization ofgoaldifficulty as a moderatorof the difficulty-performance relationship. Tubbs (1986) mad ea d istinction only between objective and subjective m easures ofdifficulty Although the difficulty perceptions category usedhere overlaps withTubbs's (1986)subjectivecategory,thisanaly-sis further differentiates the objective goal measures.

Me t hodTo test the hypothesis that the operationalization of goal difficultymoderates the relationship between goal difficulty and performance, Isubjected the 70 studies analyzed by Mento etal.(1987) to the H unter,Schmidt, and Jackson (1982) meta-analysis procedu re. I coded thesestudies according to the categories described in the next paragraphand subjected them tothe normal meta-analysis and then to m oderatortests. The studies and th e codings can be found in Table 1.My examination of these 70 studies revealed four coherent catego-ries of different operationalizations.' Two raters rated the 70 studies.

' Two additional o perationalizations of goal difficulty were ob-served from theMentoet al.(1987)studies.Afinalcoherent operation-alization of goal difficulty is labelled percen tile goals, which con-sisted of assigninggoals toperformina certain percentile of the experi-mental group. Only two studies (three effect sizes) used thisoperationalization (Erez, 1977; Motowildo, Loehr, Dun nette, 1978).In addition, two studies (Organ, 1977; Terborg, 1976) fell into the


2/9

PATRICK M. WRIGHTTable1Studies of the Relations B etween G oal Difficulty and Performance, Broken Down by Operationalization

Investigators NAssigned goal level

Locke (1966)Study1Study 2Study 3Locke, Bryan, & Kendall(1968)Study1Study 2Sales (1970)Rothkopf& Billington(1975)Cam pbell & I lgen (1976)Laporte&Nath(1976)Masters, Furman, &Barden(1977)Bavelas& Lee (1978)Study1Study 2Study 2aBavelas& Lee (1978)Study 3Study 4Dossett, Latham , &Mitchell (1979)Bassett(1979)Latham Saari (1979)Mowen, Middlemist, &Luther (1981)Mowen, Middlemist, &Luther (1981)Garland (1982)Latham & Marshall (1982)

Locke (1982)Peters, Chassie,Lindholm, O'Connor,& Kline (1982)Wofford (1982), Study1Garland (1983)Campbell (1984)Locke, Frederick, Lee, &Bobko(1984)Garland (1984)Taylor, Locke, Lee, Gist(1984)

49562370307392829632481285448304011660626260125

24712092585618171169

Self-set goal levelLocke & B ryan (1968)Locke, Cartledge, &Knerr(1970),Study 5

32354

d

8471.6581.81 966 990.919 636 467 6541.1160.101.761.0661.1281.2730.913 3441.099 585

-0.3351.1450.1630.871 793 4641.032 :3460.931 738 4 6

0.5810.189

r

.396.645.679

.44.456.422

.307.230.314

.494

.05.664.476

.500.551

.424.171.488

.284- .165.504.082

.384

.373.228.458.187

.424.351

.200

.280

.096

Investigators N dSelf-set goal level continued)

Dach ler& Mobley (1973)Dac hler& Mobley (1973)Hamner Harn ett (1974)Latham, Mitchell, &Dossett (1978)Yukl& Latham (1978)Rakestraw Weiss (1981)Latham & Marshall (1982)Matsui, Okada, &Kakuyama(1982)Latham AS teele (1983)Wood & Locke (1984)Study 2Study 3Study 4Study 5

173366807641174579148

21632914237

787 257 982 967 924 8271.280 6840.501 273 4670.671 953

Performance improvementLocke & B ryan (1967)Locke (1968)Locke & B ryan (1969)Pritchard& Curtis (1973)London Oldham (1976)Becker (1978)Strang, Lawrence, &Fowler (1978)Strang, Lawrence, &Fowler (1978)Mento, Cartledge, &Locke (1980)Study1

Study 2Locke Shaw (1984)

69204081180405050

195406212

436 549 356 5 70.129 356 364

-0.273

5240.513 289

Difficulty perceptionsAndrews Farris (1972)Steers (1975)Hall & Hall (1976)Oldham (1976)Hall & Foster (1977)Ivancevich & McMahon(1977a)Ivancevich & McMahon(1977b)Ivancevich & McMahon(1977c)Ivancevich & McMahon(1977c)

7813328342611901419038

336 350.241 2 5

4240.111 668 2 4

r

.368.128.445

.44.428.384.546

.327.248

.136.228.320.440

.216.278.158.248.065.179

.183- .135

.256

.248.144

.168.016.120.103.00

.232

.056

.32

.104Note. N=number of subjects;d=effect size in standard deviation units; andr = the correlation between goal difficulty and performance.

and agreement in codings was observed for97%of the cases. Th e ratersdiscussed the rem aining studies until they reached consensus regard-ing the category of operationalization used.One ra ther general category of goal difficulty opera tionalizatio nconsisted of studies in which goal difficultywasoperationalized in the

other category because they really could not be construed to mea-sure goal difficuhy at all. However, because of the small number ofstudies in these categories, the categories were not included in themoderator analyses.

design phase as an absolute level of performance without a m easuredreference to ab ility This general category was further differentiatedinto two categories: assigned goal level and self-set goal level.Thirty studies fell in the assigned goal level category N=2,555). Mostoften, th ese designs consisted of assigningtwoor three goalstoexperi-mental groups.The self-set goal level category consisted of15studies N= 2,207).These studies allowed subjects to self-set goal levels, but did not in-struct subjectsto do sorelativetotheir past performance (although one

might assume that the subjects did use past performance as a refer-ence). It is important to note that these two categories are by far themost common operationalizations of goal difficulty, and are most


3/9

OPERATIONALIZATION OF GOAL DIFFICULTY 229likely the categories that come to mindwhen oneth inks of the relation-ship between goal difficulty and performance.I labelled the third category of goal difficulty operationalizations performance improvement. Studies using this operationalization ofgoal difficulty normally set performance goals as a percentage of apast trial or block of trials. In all cases these studies used at least twodifferent goal levels (e.g., a10%or20%increase), and the analysis com-pared the performance of the different goal groups. Eleven studies N=1,343 fell into this category.My fourth category of goal difficulty o peratio nalizatio ns, diffi-culty perceptions, is similartowhatTubbs (1986)classifiedas subjec-tive measures ofgoal difficulty These operationalizations wereself-repo rts of intentions to perform well or percep tions of the difficulty ofthe goal. Nine studies(A =1,056 used this operationalization of diffi-culty. Because these o perationalizations varied so much, they are cov-ered in detail to illustrate the differentwaysrecent m eta-analyses (andconsequen tly th e goal setting literature) have defined goal difficulty.

Andrews and Farris (1972) measured scientists' experienced timepressure. Hall and Hall(1976)and H all and Foster(1977)used3-itemscales measuring subjects' intentions to do well. Ivancevich andMcM ahon (1977a, 1977b, 1977c) used a5-itemmeasure of goal chal-lenge with items suchas It takes a lot of effort on my par t to achievethe resu lts expected formyjob. Although no sample items were given.Steers (1975) also used a multiple-item measu re that he termed chal-lenge. Finally, Oldh am (1976) asked subord inates of a focal man agerto indicate how often My supervisor sets specific performance goalsor quotas for me to achieve. This measure was correlated with thatfocal supervisors' manager's evaluation of those sub ordinates' produc-tivity as a group .It should be noted that this category of operationalization reallydoes not measure the specific quantitative goals discussed by Locke(1968). In fact, intwocases (Hall Hall, 1976; Hall Foster,1977)theopera tionalizatio n of goal difficulty clearly is more of the Do yo urbest goal condition maligned by goal theorists. Andrews and Farris's(1972) measure of experienced time pressure would have to be consid-ered m ore role overload than goal difficulty Level of analysis issuesplague the Oldham (1976) study. Thus, from a theoretical perspectiveone could question whether or not this operationalization truly testsgoal theory.If the operationalization of goal difficulty moderates the observedrelationship between goal difficulty and performance, then these dif-ferent categories of operationalization should exhibit different meaneffectsizes.An umber of tests for m oderators are available and there islittle consensus as to which test is best. Mento et al. (1987) used twomoderator tests. The first consists of subgrouping the studies on thebasis of the suspected moderator variable and then examining the un-explained variance within each subgroup. Hunter et al. (1982) statedthat if the u nexplained variance of each of the subgroups is less thanthe unexplained variance overall, then support is shown for the mod er-ating relationship.

The second test, the regression approach (M abe West,1982;Steele& Ovalle, 1984) entails regressing the effect size on the values of thesubgroups. Sup port is shown if the m oderator variable explainsasignif-icant amount of variance in effect sizes. This test is more powerfulthan the former test. In fact, in the Woo deta l. (1987)rean alysisoftheMento data, only the regression approach was used to test for taskcomplexity as a mode rator of the goal difficulty-performance rela-tionship. To maintain consistency w ith the original data, I used bo thtests in this study.

ResultsIn the preliminary analysis using all of the studies in the

Mento et al. (1987) meta-analysis, these authors' results wereexactly replicated with an effect size of .5451, an effect

corrected for unreliability to be.5831,an observed v ariance of.1495,and an unexplained variance of.1029.However, because of the deletion of five of the studies, theoverall results were slightly different. Using the65studies to betested in the moderatoranalysisresulted in a m ean effectsizeof.5538,an effect cor rected for unreliab ility of .5906, an ob servedvariance of.1447,and an unexp lained variance of.1070.Theseresults are displayed in Table 2.The first moderator test was performed by subgrouping thestudies as mentioned above and comparing the mean effectsizes and v ariances within each subgrou p w ith the mean effectsize and variance of the sample as a w hole. It should be notedthat at this point no corrections were made for measurementerror. The rationale forthiswillbediscussedlater.Thisanalysisrevealed substantial differences in mean effectsizebetween th eassigned goal level and all other opera tionalizations. Exam ina-tion of Table 2 reveals that the assigned goal level operation ali-zation had the largest effectsize .1411)followedbyself-set goalevel (.5729), performance improvement (.3798), and difficultyperceptions (.2663). The unexplained variance was substan-tially redu ced within all bu t th e assigned goal level operation-alization.Wood et al. (1987) noted that differences in criterion relia-bilities might have accounted for the different effect sizes ob-served because of taskcom plexity.Asimilar argument m ight bemade here regarding the predictor reliabilities. Mento et al.(1987) used a mean predictor reliability of .72 for goal diffi-culty, but this reliabilitywasbased a lmost solely on the reliabil-ities reported in difficulty perception studies. Other studieseither manipulated goal difficulty or used1-itemself-reports o fa self-set goal level. In these cases, the reliability ofthepredic-tor seems unimportant.For these reasons, I made the correction for predictor unre-liability only for the difficulty perception studies, not for theother stud ies (thus assumingareliability of1).T he result of thisstrategy was to increase the estimated effect size of difficultyperceptions relative to the other operationalizations, but thisincrease did no t appreciably change th e results.As waspreviously m entioned, the subgroup ing analysisisnotextremely powerful for detecting m oderators. Thus, I used theregression approach to determin e if this categorization schemeexplained a significant amount of variance in effect sizes and ifthe m ean effect sizes within each operationalization differedsignificantly from one another. When more than two sub-groups exist, this method can be performed by dummy codingthegsubgroups as ^ - 1 variables and app lying a multiple re-

gression procedure (Cohen & Cohen, 1983). To do this, I cre-ated three dumm y coded variables to reflect the four categoriesof operationalizations (the assigned goal level, self-set goallevel, performance improvement, and difficulty perceptionscategories, dummy coded 1, 0, 0; 0, 1, 0; 0, 0, 1; and 0, 0, 0respectively). The coding provided for th e observed B weightsto reflect the contrasts of each of the first three categoriesagainst the difficulty perceptions category.I regressed the effect size on the three dummy coded vari-ables,which resulted in a multipleRof509 R^ .259,adjustedR^ .224,p


4/9

3 PATRICK M . W R I G H TTable 2Meta Analysis ResultsofRelationship Between Goal DifficultyandP erformance,Broken DownbyOperationalizationofGoal Difficulty

VariableOverallAssigned levelSelf-set levelImprovementPerception

Samplesrze71612555220713431056

Numberofstudies653015119

Meanobservedd.5538.7477.5729.3798.2663

Observedvariance

.1447.1981.0696.0356.0361

Meancorrectedd

.5906.7797.5974.3960.3414

Variancecorrectedforsamplingerror.0377.0732.0283.0334.0344

Unexplainedvariance

.1070.1249.0413.0022.0017

Percentvarianceunexplained

73.963.059.36.24.7Note,dstandsforeffect sizeinstandard deviation units.

from the performance improvement and difficulty perceptioncategories. In add ition, t he self-setgoallevel operation alizationwas significantly different from the difficulty perception cate-gory No other categories were different from one another.I performed an add itional analysis to examine the joint roleof operationalization of goal difficulty and task complexity asexplanations of variance in effect sizes. I obtained the taskcomplexity codings used in the Wood et al. (1987) study fromthe authors and comp uted two regressions. The first equationregressed the effectsizeson task comp lexity T he second regres-sion equation entered the dumm y coded variables in the firststep and task com plexity in the second step. In the first regres-sion equation task complexity explained13 of the variance ineffect sizes.^ When I controlled for the operationalization ofgoal difficulty in the first step of the regression, the am ou nt ofincremental variance explained by task comp lexity was re-duced, yet task complexity still accounted for a significant 7(p < .01) ofthevariance (totalR = .33,p < .01).Discussion

Over 20 years of research on goal setting has shown substan-tial support for goal theory This article argues, however, thatrecent cumu lations of this research have ignored th e con structvalidity issues relative to the mo stbasicconstru ct of the theory:goal difficulty The resu lts of this m oderato r analysis show that26 of the variance in effect sizes observed in g oal difficulty-performance relationships can be explained by the m anner inwhich goal difficulty was oper ationalized. In con trast. W ood etal. (1987) demonstrated that task complexity moderates thisrelationship based on its ability to explain 5.7 of the variancein the se effect sizes. If task comp lexityisviewedasamod erator,then serious consideration m ust be given to the op erationaliza-tion of goal difficulty as a moderator of the relationship be-tween goal difficulty and performance. In this study the opera-tionalization explained twice as much variance as did taskcomplexity, and , even after con trolling for task com plexity, theoperationalization ofgoal difficulty explained an incremental20 of the variance in effect sizes.The purpose of this article is not to imply that goal setting isnot an effective motivational tool, nor to deny that a positiverelationship exists between goal difficulty and performance;my purpose is to show that the way in which goal difficulty isoperationalized may have profound implications for the effectsize observed. The concept of goal difficulty seems so com-

mon-sensical that one might assume that all manipulations arethe same. his articlehasdem onstrated, however, that this sim-ply is not the case.The results showed that although all the operationalizationswere positively related to performance, these relationships dif-fered significantly in their strength. These results indicate thatperhaps operationalizations that have been cumulated in pastmeta-analyses ofgoaldifficulty have either m easured differentconstruc ts or m easured the same co nstruct with different levelsof validity Examination of these operationalizations revealsthat three of the four operationalizations (assigned goal level,self-set goal level, and performance improvement) truly repre-sent the specific difficult goals from which goal theory makespredictions. However, these operationalizations exhibited sig-nificantly different effect sizes. Examining the reason for thedifferences between these op erationalizatio ns reveals some ex-tremely im portant implications for the interpretations of past,

as well as the design of future, goal-setting research .The assigned goal level operationalization may displayhigher effect sizes than performance improvement because therange in difficulty was greater than for the other operational-izations. If easy, moderate, and difficult goals (according togroup performance norms) are assigned to individuals ran-domly,som e high-ability subjects w illhaveextremely easy goalsand some low-ability subjects will have extremely difficultgoals. This situation would lead to high-ability subjects lower-ing their level of performance to be in line w ith the goal. Forthis reason Locke and Latham (1990) stated that d o-best goalswould be m ore effective than specific easy goals (Locke, M ento,&Katcher, 1978).Under performance improvement goals, however, all sub-jects'goals would be much closer to their ability levels. Underself-set goal level operationalizations, in spite of the spuriousrelationship between goal level and perform ance attribu table toability, subjects w ould s till be expected to set goals that wouldbe neither extremely difficult nor extremely easy Thus, theseoperationalizations would be expected to have smaller rangesin goal difficulty, and thus lower observed effect sizes.^In theWoodetal.(1987) meta-analysis, task comp lexity explained5.7 of the variance ineffect sizes. That study, however, added threeunpublished studiesto theoriginal Mentoetal. (1987) meta-analysis,and these studies weren otincluded inthis analysis.T heexclusiono fthese studies,aswellas the studies mentioned in Footnote1, is thereasonforthe d ifferences in theamou nt of variance explainedb ytaskcomplexity(5.7 vs. 13 ).


5/9

OPERATIONALIZATION OF GOAL DIFFICULTY 231Table 3Regression ResultsofModerator AnalysisOperationalization B weight' Significance contrasts

Level (assigned)Level (self-set)ImprovementPerceptions

.391

.076.295

7.437.088.253

Significantly different from perceptions category at.01and significantly differentfrom improvement category at .05.Significantly different from perceptions category at.01.

Note. N= 65.Th eBweights do not match the m ean effectsizesin Table2becauseTable 2uses sample-weighted effectsizes,whereasTable 3 usesunit-weightedeffect sizes. Because of dummy coding, theBweight for the perceptions category is the residual term of the full regression equation.

Although these three operationalizations may be equallyvalid, these results have implications for research on the goaldifficulty-performance relationship. First, the greater range ingoal difficulty inherent in a goal level operationalization is use-ful in tha t it should increase th e effectsizeand thu s increase thepower for detecting differences between goal levels. Thus, re-search directed at examining variables that might moderate ormediate the relationship between goal level and performancecan most efficiently be conducted using the assigned goal leveloperationalization.However, a weakness of the operationalization is that onemust question the accuracy of including such studies in thecom putation of a percentage increase attributable to goal diffi-cultyasM entoetal.(1987)did.Forexample, M ento et al. (1987)referred to goal settingas a motivational approach for enhan c-ing productivity (p.52), and expressly stated that one purpo seof their study was . . . to know, for example, what percentageincrease in productivity m ight be expected when specific hardgoalsare usedasan organizational intervention (p.56). On thebasis of their analysis they stated tha t for goal difficulty theeffect size ^= .5813 (across all studies)isequal to a productivityincrease of11.63% (p. 76). Such a statement is misleading be-cause the effectsizethey comp uted d oes not representalevel ofperformance increase, but only a linear relationship betweengoal level and performance. In other words, a large effect sizewill be observed in an assigned goal level operationalization,but the way that goal setting causes this effect m ay be by caus-ing a num ber of subjects to decreasetheir performance (Lockeetal., 1978).With regard to the true effect size of goal difficulty for in-creasingperformance, a more accurate estimate may be muchcloser to the .3798 observed in the performance improvementoperationalization. This may support Dunnette's (1973) con-tention th at abilityis amuch m ore potent determinan t of perfor-mance than motivation. Although this does not preclude thevalue of motivational interventions, it may call for a m ore con-servative estimate of their effectiveness for increasing perfor-mance.This study demonstrated that vast differences in effect sizeshave been observed in past research b ecause of differences inthe way goal difficulty was operationalized. Future researchshould pay more attention to operationalizing goal difficulty ina way that accurately depicts the theoretical construct beingexamined. This is becoming increasingly important because avariety of motivational theories, such as expectancy theory(Campbell&Pritchard,1976;Dachler&Mobley,1973;M atsui,Okada, & Mizuguchi, 1981), NPI theory (Naylor & Ilgen,

1984), control theory (Campion & Lord, 1982; Klein, 1989),cognitive mediation theory (Garland, 1985), and goal theory(Locke et al. 1981), have attempted to explain the relationshipbetween goal difficulty and performance. These explanationswill require more theoretically specific definitions of the con-struct of goal difficulty, and these definitions may vary accord-ing to the specific the ory being tested.In conclusion, my analysis suppo rts Schwab's (1980) discus-sion of the positive value of examining an d exploring c onstructvalidity previous to or concurrently with substantive validityUntil now, however, the goal-setting literature has been vir-tually devoid of construct validation of one of the most basicconstru cts of go al-setting theory goal difficulty This studysuggeststhat the con struct o f goal difficulty needs tobetheoret-ically analyzed before it is operationally defined. Such con-struct validity consideration in the design phase of go al-settingresearch may make future meta-analyses of the goal difficulty-performance relationship better able to explore potential situa-tional moderators such as monetary incentives or individualdifferences. If so, then our understanding of substantive rela-tionships m ay increase as a result oftheincreased con cern forconstruct validity

ReferencesAndrews,FM.,&Farris,G G (1972).Time pressure andperformanofscientistsand engineers: Afive-yearpanel study.OrganizationBehavioran dHumanPerformance, 8,185-200.Campb ell, J. P, & Pritchard, R. D. (1976). Motivation theory in indus-trial and organizational psychology In M. D. D unnette (Ed.),Handbookofndustrialandorganizational psychology(pp. 63-130). Chicago, IL: Rand McNallyCampion, M. A.,&Lord, R. G. (1982).Acontrol systems conceptual-

ization of the goal setting and changing process.OrganizationalBe -havioran dHumanPerformance, 30 , 265-287.Cohen, J., & Cohen, P (1983).Applied multiple regression correlatioanalysisfo rthe behavioralsciences.H illsdale, NJ: Erlbaum.Dachler, H. P, & Mobley, M. F (1973). Construct validation of an in-strumentality-expectancy-task-goal model of work motivation:Some theoretical boundary conditions[Monograph].Journal ofAp -pliedPsychology58 ,397-418.Dun nette, M . D. (1973).Performance equals abilitya ndwhat? JtchRep. No. 40 09, Contract No. N0014-68-A-0141-003). San Diego: Of-fice of Naval Research, Personnel Training and Research Programs,Psychological Sciences Division.Erez, M. (1977). Feedback:Anecessary cond ition for the goal setting-performance relationship. Journal of Applied Psychology, 62,624-627.Garland, H. (1985). A cognitive mediation theory of task goals andhuman performance. Motivationand Emotion, 9, 345-367.


6/9

PATRICK M. WRIGHTHa ll, D. T, Foster, L. W (1977). A psychological success cycle andgoal setting: G oals, performance, and attitudes. Academ y of Manage-mentJournal, 20 , 282-290.Hall, D.T. Hall,FS.(1976). Th e relationship between goals, perfor-mance, success, self-image, and involvement under different organi-zational climates.JournalofVocationalBehavior, 9, 267-278.Hun ter, J. E., Schm idt, F L., Jackson , G. B. (1982). Meta-analysis:

Cumulatingresearchfindingsacrossstudies.Beverly Hills, CA : Sage.Ivancevich, J. M., McM ahon, J. T. (1977a). Black-White differencesin a goal-setting program.Organ izational Behavior and HumanPer-formance, 20, 287-300.Ivancevich,J.M ., McMahon,J.T.(1977b). Ed ucationas amoderatorof go al-setting effectiveness. Journal ofVocationalBehavior, 11 ,83-94.Ivancevich, J. M., McMahon,J.T.(1977c).Astudy of task-goal attri-butes, higher order need strength, and performance. Academy ofManagementJournal, 20 , 552-563.Klein, H. J. (1989). An integrated con trol theory model of work m oti-vation. cademyofManagementReview, 14 ,150-172.Latha m, G. P, Erez, M., Locke, E. A. (1988). Resolving scientificdisputes by the joint design of crucial experiments by the antago-nists: Application to the Erez-Latham dispute regarding participa-tion in goal setting.Journalof ppliedPsychology, 73 , 753-772.Locke, E. A. (1968). Towardatheory of task motivation an d incentives.Organizational Behavioran dHumanPerformance, 3,157-189.Locke, E. A., Latham, G. P (1990).Atheoryofgoal settingandtaskperformance.Englewood Cliffs, NJ: Prentice-Hall.Locke, E. A., Mento, A. J., Katcher,B.L. (1978). The interaction o fability and m otivation in performance: An exploration of the m ean-ing of moderators.PersonnelPsychology, 31, 269-280.Locke, E. A., Shaw, K. N., Sa ari, L. M., Lath am, G. P (1981). Goa lsetting and task performance: 1969-1980.Psychological Bulletin,90,125-152.Mabe, P A., West,S.G. (1982). Validity of self-evaluation of ability:A review and meta-analysis. Journal of AppliedPsychology, 67 ,280-296.

Matsui, T., Okad a, A., Mizuguchi, R. (1981). Expectancy theoryprediction of the goal theory postulate: The harder the goals, thehigher the performance.Journalof ppliedPsychology,66, 54-58Mento, A. J., Steel, R. P, Ka rren, R. J. (1987). A meta-analytic studyof the effects of goal setting on task performance: 1966-1984. Orga-nizational BehaviorandHuman DecisionProcesses, 39, 52-83.^Motowidlo, S. Loehr, V, Dunnette, M.D.(1978).Alaboratory study

of the effects of goal specificity on th e relationship b etween p robab il-ity of success and performance.Journalof ppliedPsychology, 23 ,172-179.Naylor, J. C , Ilgen, D. R. (1984). Goal-setting:Atheoretical analysisof a motivational technology InB.Staw L. L. Cumm ings (Eds.),Research inorganizational behavior Wo\.6, pp. 95-140). GreenwiCT: JAI Press.Oldham,G. R.(1976). The m otivational strategies used by supervisors:Relationships to effectiveness indicators.Organizational Behavioran dHumanPerformance, 15 ,66-86.Organ, D. W (1977). Intentional versus arousal effects of goal setting.Organizational Behavioran dHumanPerformance, 18 , 377-389.Schwab,D.P (1980). Construct validity in organizational behavior. InB.Staw L. L. Cum mings (Eds.),Research inorganizational behav-

ior Vol.2, pp. 3-43). G reenwich, CT: JAI Press.Steele, R. P, Ovalle, N. K. (1984). A review and meta-analysis ofresearch on the relationship between behavioral intentions andemployee turnover.Journalof ppliedPsychology,69 , 673-686.Steers, R. M . (1975). Task-goal attributes, achievement, and supervi-sory performance. Organizational Behavior and HumanPerfor-mance, 13 , 392-403.Terborg, J. R. (1976). The motivational components of goal setting.Journalof ppliedPsychology, 61,613-621.Tubbs, M. (1986). Goal setting: A meta-analytic examination of theempirical evidence.Journalof ppliedPsychology, 71,474-483.Wood, R. L., Mento, A., Locke, E. (1987). Task complexity as amoderator of the goal difficulty-performance relationship. Journalof pplied Psychology 73 ,416-425.


7/9

OPERATIONALIZATION OF GOAL DIFFICULTY Appendix

Studies Used for CategorizationAndrews, F M., Farris, G. F (1972). Tim e pressure and performanceof scientists and engineers: Afiveyear panel study.OrganizationalBehavioran dHumanPerformance 8 185-200.Bassett, G. A. (1979). A study of the effects of task goal and schedulechoice on work performance.Organizational Behaviorand HumanPerformance 24 202-227.Bavelas, J.B., Lee, E.S.(1978).Effects of goal level on perform ance:A tradeoff of qu antity and qualityCanadian Journalof Psychology52,219-240.Becker, L. J. (1978). Jo int effect of feedback and goal settingonperfor-mance: A fieldstudy of residential energy conservation.Journal of pplied Psychology 63 428-433.Cam pbell, D. J. (1984). The effect ofgoalcontingent payment on theperformance of a complex task.PersonnelPsychology. 37 23-40.Campbell,D.J. Ilgen,D.R. (1976). Additive effects of task difficultyand goal settingonsubsequent task performance.Journal ofAppliedPsychology 61 3\9-i24.Dachler, H. P, Mobley, W H. (1973). Co nstru ct validation of aninstrumentality-expectancy-task-goal model of work motivation:Some theoretical boundary conditions [Monograph].JoumalofAp-plied Psychology 58 397-418.Do ssett, D. L., Lath am, G. P, Mitchell, T. R. (1979). The effects ofassigned versus participatively set goals, KR, and individual differ-ences when goal difficulty is held constant.Journalof ppliedPsy-chology64 29\-29S.Erez, M. (1977). Feedback:Anecessary condition for the goal setting-performance relationship. Journal of Applied Psychology 62624-627.Garland , H . (1982). Goal levels and task p erformance: A com pellingreplication of some compelling results.Journalof ppliedPsychol-ogy 67 245-248.Ga rland , H . (1983). Influence of ability, assigned g oals, and norm ativeinformation on personal goals and performance:Achallenge to thegoal attainability assumption. Journal of AppliedPsychology 6820-30.Garland , H. (1984, August).Acognitive mediation theoryofask goalsand humanperformance.Paper presented at the 44th annual conven-tion of the Academy of Management, Boston.Ha ll, D. T, Foster, L. W (1977). A psychological success cycle andgoalsetting: Goals, performance, and attitudes.Academ y of Manage-mentJournal 20 282-290.Hall,D.T , Hall, FS.(1976). The relationship between goals, perfor-mance, success, self-image, and involvement under different organi-zational climates.JournalofVocationalBehavior 9 267-278.Hamner,WC , Harnett,D.L.(1974). Goal-setting, performance andsatisfaction in an interdependent task.Organizational Behavior and

HumanPerformance 12 217-230.Ivancevich,J.M ., McMahon,J.T.(1977a). Black-White differences ina goal-setting program .OrganizationalBehavior andHumanPerfor-mance 20 287-300.Ivancevich,J.M ., McMahon,J.T.(1977b). Educ ationas amoderatorof goal-setting effectiveness. Journal ofVocationalBehavior 1183-94.Ivancevich,J.M., McMahon,J.T.(1977c).Astudy of task-goal attrib-utes, higher order need strength, and performance.Academy of Man-agementJournal 20 552-563.Jackson , S. E., Zedeck, S. (1982). Explaining perform ance variab il-ity: Contributions of goal setting, task characteristics, and evaluativecontexts.Journalof ppliedPsychology 67 759-768.LaPorte, R. E., Nath, R. (1976). Role of performance goals in proselearning.JournalofEducational Psychology 68 260-264.

Latham,G.P, Mars hall, H . A . (1982). The effects of self-test, pa rtici-patively set, and assigned goals on the performance of governmentemployees.Personnel Psychology 35 399-404.Latham , G. P, Mitchell,T.R., Dossett, D.L.(1978). The im portanceof participative goal setting and anticipated rewards on goal diffi-culty and job performance. Journal of AppliedPsychology 63163-171.Latham , G. P, Saari, L. M. (1979). The importance of supportiverelationships in goal setting. Journal of AppliedPsychology 64151-156.Latham,G.P, Steele,T.P(1983).The m otivational effects of pa rtici-pative versus assignedgoalsetting on performance.Academ y of Man-agementJournal 26 406-417.Locke, E. A. (1966). The relationship of intentions to level of perfor-mance.Journalof pplied Psychology 50 60-66.Locke, E. A. (1968). The effects of knowledge of results, feedback inrelation to standard s, and goals on reaction time p erformance.Amer-

ican JournalofPsychology 81 566-574.Locke, E. A. (1982). Relation of goal level to performanc e with a shortwork period and multiple goal levels.Journal ofApplied Psychology67,512-514.Locke, E. A., Bryan, J. F (1967). Performance goalsasdeterminantsoflevelof performance and boredom .Journal of Applied Psychology57,120-130.Locke, E. A., Bryan, J. F (1968). Grade goals as determinants ofacademic achievement.JournalofGeneralPsychology 79 217-228Locke, E. A., Bryan, J. F (1969). The directing function ofgoalsintask performance. Organizational Behavior and Hum anPerfor-mance 4 35-42.Locke,E.A., Bryan,J.F, Kendall,L.M.(1968). Goals and intentionas med iators of the effects of mone tary incentivesonbehavior.Jour-nalof pplied Psychology 52,104-121.Locke, E. A., Cartledge,N., Knerr,C.(1970). Studies of the relationship between satisfaction, goal setting and performance. Organiza-tional Behavioran dHumanPerformance 5 135-158.Locke, E. A., Frederick, E., Lee, C , Bobko, P (1984). Effects ofself-efficacy, goals, and task strategies on task performance.Journalo f pplied Psychology69, 241-251.Locke, E.A., Shaw,K.N.(1984). Atkinson's inverse-U curve and themissing cognitive variables.PsychologicalReports 55 403-412.London, M., Oldham,G.R. (1976). Effects of varyinggoaltypes andincentive systems on performance and satisfaction.Academy of Man-agementJournal 19 537-546.Masters, J. C , Furm an, W, Barden , R. C. (1977). Effects of achieve-ment stan dards, tangible rewards and self-dispensed achievementevaluations on children's task master. Child Development 48

217-224.Ma tsui, T, Okad a, A., Kaku yama, T. (1982). Influence of achieve-ment need ongoalsetting, performan ce, and feedback effectiveness.Journalof ppliedPsychology 67 645-648.Mento, A. J., Cartled ge, N. D , Locke, E. A. (1980). Maryland vs.Michiganvs.M innesota: Ano ther look at the relationship of expec-tancy and goal difficultytotask performance.O rganizational Behav-ioran dHumanPerformance 25 419-440.Motowidlo, S. Loehr, V, Dunnette, M.D.(1978).Alaboratory studyof the effects ofgoalspecificity on the relationship between prob abil-ity of success and performance.Journalof ppliedPsychology 23172-179.Mowen, J. C , Midd lemist, R. D, Luther, D. (1981). Joint effects ofassigned goal level and incentive structure on task performance: Alaboratory study.Journalof ppliedPsychology 66 598-603.


8/9

4 PATRICK M. WRIGHTOldham,G.R.(1976). The motivational strategies used by supervisors:Relationships to effectiveness indicators.Organizationat Behavioran dHumanPerformance IS 66-86.Organ, D.W(1977). Inten tionalvs.arousal effects ofgoalsetting.Orga-nizational Behaviorand Human Performance 18 377-389.Peters,L. H.,Chassie,M. B.,Lindholm, H. R.,O'Connor,E. J., &Kline,C. R.(1982). The joint influenceofsituational constraints

and goal setting on performance and affective outcomes. Journal ofManagement, 8 7-20.Pritchard,R.D., Cu rtis, M.I.(1973). The influenceofgoalsettingand financial incentiveson task performance.O rganizationat Behav-ioran dHumanPerformance 10 175-183.Rakestraw , T. L. Jr., Weiss,H.M. (1981). The interaction of socialinfluenceandtask experienceon goals, performance,andperfor-mance satisfaction. OrganizationatBehaviorand HumanPerfor-mance 27 326-344.Rothkopf, E. Z., Billington, M. J. (1975). A two-factor model of theeffect o f goal-descriptive d irection s on learning fromtext.Journal ofEducationalPsychotogy 67 192-204.Sales, S. M. (1970). Some effectsofroleoverload and role underload.Organizationat Behavioran dHumanPerformance 5 592-608.Steers,R.M. (1975). Task-goal attributes, achievement, and supervi-sory performance.Organizationat BehaviorandHumanPerfor-mance 13 392-403.

Strang,H. R.,Lawrence,E. C, &Fowler,PC. (1978). Effectsofas-signed goal levelandknowledgeofresults on arithm etic com puta-tion: A laboratory study.JournalofAppliedPsychology 63 29-39.Taylor, M .S.,L ocke, E. A., Lee, C, Gist, M. (1984). TypeAbehaviorand faculty research productivity: What are the mechanisms? Orga-nizational BehaviorandHumanPerformance 34 402-418.Terborg,J. R.(1976).Themotivational compon entsofgoal setting.

JournalofAppliedPsychology 61 dX3-621.Wofford, J. C. (1982). Experime ntal testsofthe goal-energy-effortre-quirement theoryofwork motivation.Psychological Reports501259-1273.Wood,R.E., Locke, E. A. (1984).The effectso fself efficacy onaca-demicperformance. Unpublished manuscript. UniversityofNewSouth Wales, Australia.Yukl,G. A., &Latham,G. P (1978). Interrelationships amongemployee participation, individual differences, goal difficulty, goalacceptance, goal instrumentality and performance.PersonnelPsy-chology 31 305-323.

Received April 26,1989Revision received October 23,1989

Accepted November6,1989

CorrectiontoPaese and SwitzerA replication of Paul W Paese's and Fred S. Switzer's study, Validity Generalization and

Hypothetical Reliability Distributions: A Test of the Schmidt-Hunter Procedure JournalofAppliedPsychology,1988, Vol. 73, No. 2, pp. 267-274) found artifactual variance estimatesconsiderably smaller than those in the original study. The discrepancies were traced to an errorin the original Monte Carlo computer program. (We thank Frank Schmidt for calling to ourattention the possibility of such an error.) As in the original study, true reliability distributionsand sample sizes were systematically varied to observe their effects on variance estimatesproduced by the Schmidt and Hunter noninteractive and interactive validity generalization(VG) equations. Consistent with the original study, the results of the replication indicated thatartifactual variance was overestimated by the noninteractive equation. Contrary to the originalstudy, the interactive equation provided estimates of artifactual variance that were unbiased.Moreover, the replication supported our original conclusion that the use of hypothetical reli-ability distributions can lead to inaccurate conclusions in VG research. (Interested readers canobtain revised tables andfiguresfrom Paul W Paese, Department of Psychology, University ofMissouri-St. Louis, 8001 Natural Bridge Road, St. Louis, Missouri 63121.)


9/9

Documents

Goal Diffulty (3)