More Effective Choice in the Prisoner's DilemmaAuthor(s): Robert AxelrodSource: The Journal of Conflict Resolution, Vol. 24, No. 3 (Sep., 1980), pp. 379-403Published by: Sage Publications, Inc.Stable URL: http://www.jstor.org/stable/173638 .Accessed: 15/03/2011 23:39Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=sage. .Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.Sage Publications, Inc. is collaborating with JSTOR to digitize, preserve and extend access to The Journal ofConflict Resolution.http://www.jstor.orgMoreEffectiveChoiceinthe Prisoner'sDilemma ROBERTAXELROD Institute ofPublicPolicyStudies The University ofMichigan This study reports and analyzes the results of the second round of the computer tour- nament for the iterated Prisoner's Dilemma. The object is togain a deeper understanding of how to perform well in such a setting. The 62 entrants were able to draw lessons from the results ofthe first round and were able todesign their entries totake these lessons into account.Theresults of the second round demonstrate a number of subtle pitfalls which specific types of decision rules can encounter. The winning rule was once again TIT FOR TAT, the rule which cooperates on the first move and then does what the other player did on the previous move. The analysis of the results shows the value of not being the first to defect,ofbeingsomewhatforgiving,but alsotheimportanceofbeing provocable.An analysisofhypotheticalalternative tournaments demonstrates therobustness of the re- sults. THETOURNAMENTAPPROACH Thisarticlereportsandanalyzestheresultsof thesecondroundof the ComputerTournamentforthePrisoner'sDilemma.Theobjectofthe studyis togaina deeperunderstandingofhowtoperformwell in sucha Prisoner'sDilemmasetting. ThePrisoner'sDilemmagameisjustifiablyfamousasanelegant embodimentofthetensionbetweenindividualrationality(reflectedin theincentiveofbothsidestobeselfish)andgrouprationality(reflected inthehigherpayofftobothsidesformutualcooperation).Atypical AUTHOR'SNOTE:Iwouldliketothankallthepeoplewhoseentries madethis tournamentpossible.Their names are given in Table 2.1 wouldalso like tothank John Chamberlin, Michael Cohen, and William Keech for their helpful suggestions concerning thedesignandanalysisofthetournamentand JohnMeyer forhis helpingetting the submissions in machine-readable form.For its support of this research, I owe thanks to theInstitute ofPublicPolicyStudies. JOURNALOFCONFLICTRESOLUTION,Vol.24 No.3,September1980 379-403 ?1980 SagePublications,Inc. 379 380JOURNALOFCONFLICTRESOLUTION payoffstructureisillustratedinTable1. Whentheinteractionbetween thetwoplayersisiterated, thedefectingchoiceisnolongernecessarily thebestchoice.Withiteration, bothsideshaveanongoingrelationship withaninformativehistoryandanimportantfuture.Thismakes possiblesophisticatedstrategieswhichusethishistory. ThebestwaytoplayaniteratedPrisoner'sDilemmaofuncertain durationdependsonwhatdecisionruletheotherplayerisusing.In otherwords,thereisnoonerulewhichisoptimalindependentofthe otherplayer'srule.Toseewhythisisso,supposearuleisnice(i.e.,is neverthefirsttodefect).Thenunconditionaldefectionwoulddobetter iftheotherplayerisusingunconditionalcooperation.Alternatively, supposearuleisnotnice.Thenunconditionalcooperationwoulddo betteriftheotherplayerisusingthefollowingrule:cooperateuntilthe otherdefects,andthenalwaysdefect.' Giventhatthestrategyof theotherplayeris important,a goodwayto studyeffectivechoiceistoarrangeasituationinwhicheachplayeris tryingtodowellforhimselfandeachplayerknowsin advancethatthe otherplayersareintelligentand are alsotryingtodowellfor themselves. Thisisthetournamentapproach. Inthefirstroundofthetournament,gametheoristswereinvitedto submitstrategiesforplayingtheiteratedPrisoner'sDilemma.These strategieswerecodedascomputerprogramswhichhadasinputthe historyofthepairwiseinteractionandhadasoutputthechoicetobe madeonthecurrentmove.Eachofthe14 entrieswaspairedwitheach otherentry.Asannouncedintherulesofthetournament,eachentry wasalsopairedwithitsowntwinandwithRANDOM,a programthat randomlycooperatesanddefectswithequalprobability.Thewinnerof thefirstroundofthetournamentwasTITFORTAT,submittedby ProfessorAnatolRapoportbytheDepartmentofPsychologyofthe UniversityofToronto.TITFORTATis thestrategywhichcooperates onthefirstmove,andthendoeswhatevertheother playerdidonthe previousmove.Thisfirstroundofthecomputertournamentforthe iteratedPrisoner'sDilemmaled notonlytoa numberof insights intothe characteristicsofsuccessfulrulesbutalsohighlighteda varietyof subtle thingsthatcangowrongwhentwosophisticatedstrategiesinteract.The rationaleforthetournamentapproachandadetailedanalysisofthe firstroundisgiveninAxelrod(1980). 1.To be precise, this requires that the length of the game is sufficiently great relative to thespecificpayoffmatrix.Thisconditionismet forthetournament.Fordetailssee Axelrod(1979). Axelrod/PRISONER'SDILEMMACHOICE381 TABLE1 PayoffMatrix forEach Move ofthePrisoner's Dilemma Column Player CooperateDefect Cooperate3,30,5 RowPlayer Defect5,01,1 NOTE:The payofftothe row player is given first in each pair ofnumbers. Asecondroundofthetournamenthasnowbeenconducted.The resultsofthesecondroundprovidemuchbetterinsightintothenature ofeffectivechoicein thePrisoner'sDilemmabecausetheentrantstothe secondroundwereall giventhedetailedanalysisofthefirstround.This analysisincludeddiscussionofavarietyofsupplementalrulesthat wouldhavedoneverywellintheenvironmentofthefirstround.This meansthattheywereawareoftheoutcomeofthefirstround,the conceptsusedtoanalyzesuccess,andthepitfallsthatwerediscovered. Moreover,theyeachknewthattheothersknewthesethings.Therefore thesecondroundpresumablybeganatamuchhigherlevelof sophisticationthanthefirstround,anditsresultsshouldthereforebe thatmuchmorevaluableasa guidetoeffectivechoiceinthePrisoner's Dilemma. Thesecondroundisalsoadramaticimprovementoverthefirst roundin sheersizeofthe tournament.Theresponsewasfar greaterthan anticipated.Therewere62entriesfromsixcountries.Thecontestants werelargelyrecruitedthroughannouncementsin journalsforusersof smallcomputers.Thegametheoristswhoparticipatedin the first round ofthetournamentwerealsoinvitedtotry again.Thecontestantsranged froma10-year-oldcomputerhobbyisttoprofessorsofcomputer science,economics,psychology,mathematics,sociology,politicalsci- ence,andevolutionarybiology.Thecountriesrepresentedwerethe UnitedStates,Canada,GreatBritain,Norway,Switzerland,andNew Zealand. Thesecondroundprovidesachancebothtotestthevalidityofthe themesdevelopedintheanalyisisofthefirstroundandtodevelop additionalnewconceptstoexplainsuccessesandfailures.Theentrants alsodrewtheirownlessonsfromtheexperienceofthefirstround.But 382JOURNALOFCONFLICTRESOLUTION different people drew different lessons. What is particularly illuminating inthesecondroundisthewaythe entries basedondifferent lessons actually interact. Finally,thisreport is able toevaluate therobustnessoftheresults through the construction of alternative hypothetical tournaments. The evidence strongly supports the conclusion that, under a wide variety of environments, TITFORTATis ahighly successfuldecisionrule. THEWINNER TITFORTATwasthesimplestprogramsubmittedinthefirst round, and it won the first round. It was the simplest submission in the second round, and it won the second round. Even though all the entrants to the second round knew that TIT FOR TAT had won the first round, noonewas able todesign an entry thatdidanybetter. TIT FORTATstarts with a cooperative choice, and thereafter does what the other player did on the previous move. This decision rule was known to all of the entrants to the second round because they all had the report oftheearlier round showing that TIT FORTATwas the most successful rule sofar. They hadread thearguments abouthow it was known to elicit a gooddegree of cooperation when played with humans, howitisnotveryexploitable,howitdidwellinthepreliminary tournament,andhowitwonthefirst round.Thereportonthefirst roundalsoexplainedsomeofthereasons foritssuccess,pointingin particular toits property ofnever being the first todefect ("niceness") anditspropensitytocooperateafter theother player defected ("for- giveness" with theexceptionofa singlepunishment). Thesecondroundattractedover60entries,butonlyoneperson submitted TIT FOR TAT. This was Anatol Rapoport, who submitted it the first time, even thoughan explicit tournament rule allowed anyone tosubmit any program, evenoneauthored by someoneelse. THEFORMOFTHETOURNAMENT The second round of tournament was conducted as a round robin so that each entry was paired with each other entry. As in the first round, each entry was also paired with its own twin and with RANDOM.As Axelrod/PRISONER'SDILEMMACHOICE383 before, the payoff matrix was the one given in Table1. Once again no entry was disqualified for exceedingtheallottedtime. Asannounced inthe rules, the length of the games was determined probabilistically with a .00346 chance of ending with each given move. Thisparameter waschosensothattheexpectedmedianlengthofa game would be 200 moves. In practice, each pair of players was matched five times, and the lengths of these five games were determined once and for all by drawing a random sample. The resulting random sample from theimplieddistribution specifiedthat the fivegames foreach pair of players would be of lengths 63, 77, 151, and 308 moves. Thus the average length of a game turned out to be somewhat shorter than expected at 151 moves. Since no one knew exactly when the last move would come, end- gameeffects were successfully avoidedin thesecondround. THECONTESTANTS Noneofthepersonalattributesofthecontestantscorrelated significantly with the performance of the rules. The professors did not dosignificantlybetter than theothers, nor did theAmericans. Those whowrote inFORTRANrather than BASICdid not dosignificantly better either, even though the use of FORTRANwould usually indicate accesstosomethingmorethanabottom-of-the-linemicrocomputer. TherankingoftheentriesisshowninTable2alongwithsome informationaboutthecontestants andtheprograms. On average, short programs did not do significantly better than long programs, despite the victory of TIT FOR TAT. But on the other hand, neither did long programs (with their greater complexity)do any better than short programs. So just as happened in round one, the features of the entries which do not directly relate to their mode of performance do not account for their relative successor failure. Thedetermination of justwhat doesaccountforsuccess is no easy taskbecause there are over amillionmovesin thetournament. THEREPRESENTATIVES Since there are 63 rules in the tournament and 3969 ways the rules can bepaired, theround robin tournament matrix has almost20 times as 384JO URNA L OFCONFLIC T RESOL UTION TABLE 2 TheContestants /. angtiage CountryDiscipline(FOR TRANLengthof RankNaine(if'not (J.S.)(if jaculty) or BASIC) Progrant* IAnatolRapoport(CanadaPsychologyt'5 2DannyC. ChampionF16 3OttoBorufsenNorwayF77 4RobCaveF 20 5WilliamnAdams B22 6Jim Graaskamp & t23 KenKatzen 7Herb Weiner t31 8Paul D.HarringtonI112 9T.NicolausTidemanEconomics1;38 & P. Chieruzzi 10Charles KluepfelB59 1 1Abraham Getzler 19 12FrancoisLeyvrazSwitzerlandB29 13Edward White,Jr. t16 14Graham EatherleyCanada112 15Paul L. Black 122 16Richard Hufford 145 17Brian Yamauchi B32 18JohnW. Colbert163 19Fred Mauk 163 20RayMikkelson PhysicsB27 21GlennRowsam 136 22ScottAppold 341 23Gail Grisell B10 24J.Maynard SmithUnitedKingdomBiologyt9 25TomAlmy t142 26D.Ambuelh& t23 K. Kiekey 27Craig leathers B48 28Bernard GrofmanPol.ScienceF 27 29JohannJossSwitzerlandMathematicsB 74 30JonathanPinkleyF 64 31RudyNydegger PsychologyF 23 32RobertPebley B13 33RogerFalk & B117 JamesLangsted 34NelsonWeidermanComputerSci.I 18 35RobertAdams B43 36RobynM. Dawes&Psychology F 29 Mark Batell 37GeorgeLefevre B10 38Stanley1F.Quayle F44 Axelrod/PRISONER'SDILEMMACHOICE385 Table 2 (Continued) Language CountryDiscipline(FORTRANLengthof RankName(if not U.S.)(if faculty)or BASIC)Program* 39R. D. AndersonF44 40Leslie DowningPsychologyF33 41GeorgeZimmermanF36 42Steve Newman F5 1 43MartynJonesNew ZealandB152 44E.E.H. ShurmannB32 45HenryNussbacher B52 46DavidGladsteinF28 47MarkF. Batell F30 48David A. SmithB23 49Robert LeylandNew ZealandB52 50MicharlF. McGurrinF78 51HowardR. HollanderF16 52JamesW. FriedmanEconomicsF9 53GeorgeHuffordF41 54Rik SmoodyF6 55Scott Feld SociologyF50 56Gene SnodgrassF90 57GeorgeDuisman B6 58W. H. Robertson F54 59HaroldRabbie F52 60JamesE. Hall F31 61EdwardFriedland F84 62RANDOM F(4) 63RogerHotz B14 *LengthisgivenintermsofthenumberofinternalstatementsintheFORTRAN versionoftheprogram.Aconditionalinstructioniscountedastwointernalstate- mentshere,althoughitwascountedasonlyoneinstructioninthereportofthe first round. many numbers to report as in the first round which had only 15 rules. To compress thetournament matrix, theaverage scoreofeachrule with eachotherrule isshownin Table 3as a single digit according tothe followingcode: 1:less than100 points 2:100-199.9 points(151 pointsis totalmutual defection) 3:200-299.9points 4:300-399.9points 5:400-452.9points 6:exactly453points(totalmutual cooperation) 7:453.1-499.9points 8:500-599.9 points 9:600or more points. 386JOURNALOFCONFLICTRESOLUTION While Table 3 can give some idea of why a given rule scored as it did, the amount of detail is overwhelming. Therefore, a more parsimonious methodisneededtomakesenseofthe results.Fortunately,stepwise regression provides such a method. It turns out that just five of the rules can be used to account very well for how well a given rule did with the entire set of 63. These five rules can thus be thought of as representatives ofthe full set in the sense that the scores a given rule sets with them can be used topredict the average score therule getsover the fullset. The formula for thepredicted tournament scoreis: T = 120.0 + (.202)S6+(.198)S30+(.110)S35+(.072)S46+(.086)S27 where Tis the predicted tournament score of a rule, and Sj is the score whichthatrulegetswiththe jth rule. Thisestimateofthetournamentscorescorrelatedwiththeactual tournament scores at r = .979. This means that 96% of the variance in the tournament scores is explained by a rule's performance with just the five representatives. TITFORTAT'svictory in the tournament can be explainedby its goodscores with all five of the representatives. Recall that 453 points is what is attained from unending mutual cooperation. TIT FOR TAT got the followingscores with the fiverepresentatives: S6 = 453; S30= 453; S35=453; S46=452; and S27= 446.Using these as the standard of com- parison we can see how other rules did in the tournament by seeing how much worse (or better) they did with the five representatives compared to how TIT FOR TAT did with them. This display is provided in Table 4 andwill formthe basis ofthe rest ofthe analysisofthisround. Alsoprovided in Table 4 are actual tournament scores for each rule and the residual which is the difference between the actual tournament scoreandthepredictedtournamentscore.Noticethatwhilethe tournament scores cover a range of several hundred points, the residuals areusuallysmaller than10 points,indicating againhowwell thefive representativesaccountfortheoverallperformanceoftherules. Another interesting feature of the residuals is that the top-ranking rules tend tohave the largest positive residuals indicating that they do better than most of the rules on the limited aspects of the tournament which are notaccountedforby the fiverepresentatives. We are now ready to use the representativesto help answer the central questionsofwhat worked and why. Axe/rod /PRISONER'SDILEMMACHOICE387 TABLE 3 TournamentScores (See text for legend) >1>6>? I 606666?6566'.666?, '635!.>666! 6i~565 106(,66065(600f>>06? ?.6 >44?`1442 36660066>656>66606?653?~ 66?W5656~106661I 6651>661 65?>655 6 4 44S444452 3 66666656 > ,(666> 65'>6665?5?5 S0666?l6?5?- (6?'.?65 ~ 6554 44454- 452 466666 ?65~641'666f1~~f,0~( >" '66>6>! ')666!,66>>,?166>1>14>'~44>1?4> 6 66666(>656' '>6066 >6>4 1,6'6660 ?66 '6600!'6651 '66 66>6>41 (494 442?2; 6 666(6 l6561'',666?'>4~(660(Y6661?6>(66661>O-5,521 2)6666>6',656!'6661(>6.4?66661 '6 660!>066 1>61- .,6452I7 1)66666 '>6561> >i66(~6??14(1'>666' >6 >1 '>66( 1 66?!' (1066(U(SW;13' >7 I16 66666 >)6561 !>6661'?~6> >4?'>666 '>6>!6'>1>666!(661?~>66!fI4~~~4s4 1 266666,?>6561' '>6661> 1641> >6666;41,'0666>,(66>1.'>0666'>>6>->614 4 4>4 ( 3666 6?,'>6461>'>661?46>>!,'>6664'>46>!> '>,066!'.664!,'>6!601 '3>4>6 6>4 445?42 1566666>66566l>666!.46>!6664')46?~ '>666!. > 6641,'>666>?6?44>61?44353'6>52 1766666'>6466>,666!' 46>>!'6664>46?1>'>6661, '64?,>6666( 1 1. (955674>4>S64> 7, 7'>4774.5 1~744 2'!6666666461> 1>6661' 46>~('1,'6664'>46>!> '>60661'>664?>666(1 7 166666(6S6>61 6666146>>!>,>,664>14646'>6666'.664!.6>666e,654~-53 776666666?6??60661. ?6>!'>(6664 '> 64?'>6661'66>!'066?>.~>6>3,>61-444>>>42 7366661.f6461'?666646>??('666 3'>46?S6o666!6 64?>666!?63??6125I61?34 2466661>66461'(6661.46>!'>(f666>>46>>e6661, (,6641> '666't16>1-4?6444 3433 7666666664666666146?>!>6664046>6"06661(.664,?6666t63~145-4 2775557 66>>1?6? >1>5-77>4>>'83>>6'>>'>>>41(7 54 4>4 7666666?6636? 0666?46????i666?0 ?656 6?666!.>664!>66664'63352352 795 53575?5555>1477-7 '>> -4437I4174532 31!6666?6~(6560?>666?46"???~666-464656?>,666?f '664>f. "666'.46>?16? 63~-'-42- >1> 4>2 3SI6666?'>6466(6661'(6>4??>66 6-'i,62e6>6?6f'>66 67>)666'!6,34>6?44>24>42 3 266666665 66(666?616?~?(',6W66>'466'~666?,6 646?666662643265,24- 336666665616'?6666>6 '6(>Ib 666f)64!.4>666?'>6661.06601. 364>46?64I6?) 346666666466666664 6??6(666> 6S?6?6>(6661. >,664>' ?666'.16?~I4>-6>? >(6>3 3 5666666656?66666?636>,666-4'>6636!(>666!'(>6???h6666f66942~ 3666666?646?t6666?46??O'666>(>46?1. '>666!'>664??6666053462S6>2 >36?5 3 766666>646?"66616 46>>!>6664')46>';f,666!,(.6641.16?6>'>6>?65>115>> 3666666(>646?(>666?466?6',666'>465?6661'(>6?4??666162>1S>71. - 71>3 35555553 545 53~577646>6?>6?46>5>4?6444956? I 1 2'6''23> >6 4 0666666646?666614 6>6'>666'(>465>6 '>666!''664"6666"66?-4?6?> (6> >33~ - 416666?665 66?666?46564",666>6564? >666?",664'.?666604I35.,4254 426666?6646606666?46>>?t>666>'>4 6.41 "6661.t"66 4>f' 56?()6> 161>4>6?>2;323 43666660>64666666?46>4?'~66?4 ?4666",666>' ",663"?666>',6 4>>4 66>8 4 4 2- 1 446666?665 666666?3664?6666?4(>66>6'6666.>,662!'?66664 6 4446?39-,42 242 4 56666?606566>0666?6>3653>666664>>563.'>666t.",667??6666263 >7I>6>32 4657 557?,555545 7 5>6 714>697>1>6>42397'72956o792> ,~71~17 454 476666?66 36006666146564"i666, >>66'1'666>'>663616'?66,-63'639;-3>, >6? 4655555451577775-474;43 3 73>77 44444>>35U '.>;44436.3 495 5 554 ~5555 5 5768 ~4557415 4353134 364619834>~I>6?45667>- (5> 5055 75S44~5575>55 -5t754754337-2.7A7?>1734>>4>-4-7>365 443__344 5!155573;1455555577-.755 1,'>666?666?0'>663?(6666?>66?'1 >639>1(>21 -1 5 355564?655>>?637>(343>531,4266>4>~2 133-4I3- 1>71? .,i2>9!'44562-1 545565535>455;>1'-7-37566j ?74657644(17!7444,)3 >64>>2>>2-?1(63?4I3 5544434(354435572>~44>7346443?2'>4>44666314??1>6 39>44123,4 5 7 44524227174> 4457>1i 442>1)4 99>4614 421(9 1 9>19>(9 3>24(6?4(j38>34924 564537>>243">677'3564716576?I'54,4226>-1312S42>;53 5956 23424563>I?6657> 2>2566>- >4> 2 6> 2-6 4 453>7>2-22331>>(663>66 >6>>9>' (66 I 6!144 73422734.1,447?1274??46'324??24>>?>>>?>??6.4>47129>',44>1271 >3 6244?24K 122>124547-2242>>44774421(7;22211>12>- 6>324(1742>4?6>314>234 6333321>253!;(433?1>52>>>3 >3>37-2>737',4232>>>>22?6?32'>3?23>9.616>2>6>- 388JOURNALOFCONFLICTRESOLUTION TABLE 4 PerformanceoftheRules Performancewith Representatives (Pointslostrelative toTIT FORTAT) Rev.StateTran- TournamentRuleTransitionRuleTesterquilizer RankScore6(30)35(46)(27)Residual 1434.730000013.3 2433.8800012.02.013.4 3431.7700006.610.9 4427.760001.225.08.5 5427.1000015.016.68.1 6425.6000001.04.2 7425.4800003.64.3 8425.461.037.216.61.01.613.6 9425.07000011.24.5 10424.9400026.410.66.3 1 1422.8300084.810.28.3 12422.660005.8---1.21.5 13419.6700027.061.45.4 14418.77000050.41.6 15414.110009.452.0-2.2 16411.753.6-26.841.23.4-22.4---11.5 17411.590004.061.4--4.3 18411.081.0-2.0--.87.0-7.8-10.9 19410.453.0-19.6171.83.0-14.23.5 20410.3100018.068.0-4.0 21410.2800020.057.2-4.9 22408.55000154.631.8.9 23408.11000067.4-7.6 24407.79000224.656.07.2 25407.011.02.2113.415.033.62.5 26406.95000059.6--9.4 27405.908.0-18.6227.85.614.08.9 28403.970003.01.4-17.2 29403.134.0-24.8245.04.0-3.04.4 30402.9000074.054.4--8.6 31402.16000147.4-10.0---9.6 32400.75000264.252.42.7 33400.52000183.6157.45.7 34399.98000224.641.6-1.9 35399.60000291.0204.816.5 36399.31000288.061.43.7 37398.13000294.058.42.7 38397.70000224.684.8-.4 39397.661.02.654.42.046.6-13.0 40397.13000224.672.8--2.0 Axelrod/PRISONER'SDILEMMACHOICE389 Table 4(Continued) Performance with Representatives (Pointslostrelative toTIT FORTA T) Rev. StateTran- TournamentRuleTransitionRuleTesterquilizer RankScore6(30)35(46)(27)Residual 41395.33000289.0-5.6-6.0 42394.02000224.674.0-5.0 43393.01000282.055.8-3.5 44392.54000151.4159.2-4.4 45392.4100025 2.644.6-7.2 46390.891.073.0292.01.0-.416.1 47389.44000291.0156.82.2 48388.927.8-15.6216.029.855.2--3.5 49385.002.0--90.0189.02.8101X0-24.3 50383.171.0--38.4278.01.061.8-9.9 51380.95135.6-22.0265.426.829.816.1 52380.49000294.0205.2-2.3 53344.171.0199.4117.23.088.4-17.0 54342.89167.6-30.8385.042.429.4-3.1 55327.64241.0-32.6230.2102.2181.6-3.4 56326.94305.0-74.4285.273.442.0-7.5 56309.03334.874.0270.273.042.28.4 58304.62274.0--6.4290.4294.06.0-9.3 59303.52302.0142.2271.413.0-1.01.8 60296.89293.034.2292.2291.0286.018.8 61277.70277.0262.4293.076.0178.817.0 62237.22359.2261.8286.0114.490.2-12.6 63220.50311.6249.0293.6259.0254.0-16.2 PROPERTIESOFSUCCESSFULRULES NICENESS Asin thefirstround,it paidtobenice.A "nice"rule is onewhichwill neverbethefirsttodefect.Beingthefirsttodefectwasusuallyquite costly. Morethanhalfoftheentrieswerenice,soobviouslymostofthe contestantsgotthemessagefromthefirstroundthatit didnotseemto paytobethefirsttodefect. Inthesecondround,therewasonceagainasubstantialcorrelation betweenwhetherarule wasniceandhowwellit did.Of the top 15 rules, 390JOURNALOFCONFLICTRESOLUTION allbutonewerenice(andthatonerankedeighth).Ofthebottom15 rules,allbutonewerenotnice.Andtheoverallcorrelationbetween whetherarulewasniceandits tournamentscorewasasubstantial.58. Table4showsthepatternveryclearlyinthescoreswiththefive representatives.Thefirstthreerepresentativesarethemselvesnice.All ofthenicerulesgot453pointswitheachofthesethree,sothenicerules lostnopointscomparedtohowfirst-placedTITFORTATdidwith them.TheruleswhichwerenotnicegenerallydidnotdoaswellasTIT FORTATdidwiththesefirstthreerepresentatives,asshownbythe predominanceofpositiveovernegativenumbersin thesethreecolumns ofTable4. Togiveanexample,thebestoftheruleswhichwasnotnicewas submittedbyPaulHarringtonandrankedeighth.Thisruleis a variant ofTITFORTATwhichhasacheckforRANDOM,andawayof gettingoutofalternatingdefections(echoeffects),andalsoa methodof seeingwhatit cangetawaywith.It alwaysdefectsonmove37andwith increasingprobabilityafterthatunlesstheotherplayerdefectsimmed- iatelyafteroneofthesedefections,inwhichcaseitnolongerdefects randomly.ItdidnotdoaswellasTITFORTATwithanyofthefive representatives,butitsufferedmostfromthesecondrepresentative. Withthatentryitgot37.2pointslessthanTITFORTATdid.This secondrepresentativeisREVISEDSTATETRANSITION,modified fromthesupplementaryruleofroundoneandsubmittedinroundtwo byJonathanPinkley.REVISEDSTATETRANSITIONmodelsthe otherplayerasaone-stepMarkovprocess.It makesits ownchoicesoas tomaximizeitsownlong-termpayoffon the assumptionthatthismodel iscorrect.AsHarrington'sruledefectedmoreandmore,theREVISED STATETRANSITIONrulekeptarunningestimateoftheprobability thattheotherwouldcooperateaftereachof thefourpossibleoutcomes. EventuallyREVISEDSTATETRANSITIONdeterminedthatitdid notpaytocooperateaftertheotherexploitedit,andsoonthereafterit alsodeterminedthatitdidnotevenpaytocooperateafteramutual cooperation. Soevenif theotherruleis willingtoacceptsomedefections,oncethe limitofitstoleranceisreached,itishardtoconvinceit thatone'sways havebeenmended.Whilesomeoftheotherruleswhichwerenotnice didinfactmanagetodobetterthanTITFORTATwithREVISED STATETRANSITION,theserulestendedtodomuchworsewithsome oftheotherrepresentatives. Axelrod/PRISONER'SDILEMMACHOICE391 PROVOCABILITY Aswehaveseen,it paidtobenice.Butwhatdistinguishesamongthe majorityoftheruleswhichwerenice? Toanswerthiswecanexaminetheperformanceof theniceruleswith thetworepresentativeswhichwerenotnice.Theanalysisofthenice rulescanbenarroweddowntotheirperformancewiththesetwo representativessincetheotherthreerepresentativesare themselvesnice, andallnicerulesdoequallywellwithothernicerules.2Thetwo representativeswhichwerenotnice,IshallcallTESTERandTRAN- QUILIZER. TESTERwassubmittedbyDavidGladsteinandcamein 46thin the tournament.Theruleis unusualin thatit defectsontheveryfirstmove inordertotesttheother'sresponse.Iftheotherplayereverdefects,it apologizesbycooperatingandplayingtit-for-tatfortherestofthe game.Otherwise,it defectsasmuchaspossiblesubjecttotheconstraint thattheratioofitsdefectionstomovesremainsunder.5,notcounting thefirstdefection.Thismeansthatuntiltheotherplayerdefects, TESTERdefectsonthefirstmove,thefourthmove,andeverysecond moveafterthat.TESTERdidagoodjobofexploitingseveral supplementaryruleswhichwouldhavedonequitewellintheenviron- mentof the first roundof the tournament.Forexample,TITFORTWO TATSis therulewhichdefectsonlyaftertheotherplayerdefectsonthe precedingtwomoves.ButTESTERneverdoesdefecttwicein a row.So TITFORTWOTATSalwayscooperateswithTESTER,andgetsbadly exploitedforitsgenerosity.NoticethatTESTERitselfdidnotdo particularlywellinthetournament,averagingonly391pointstocome in46th.Itdid,however,providelowscoresforsomeofthemoreeasy- goingrules. AsanotherexampleofhowTESTERcausesproblemsforsomerules whichhavedonewellinthefirstround,considerthethreevariantsof LeslieDowning'soutcomemaximizationprinciple.Thereweretwo separatesubmissionsoftheREVISEDDOWNINGprogramwhich lookedsopromisinginroundone.ThesecamefromStanleyF.Quayle andLeslieDowninghimself.Aslightlymodifiedversioncamefroma youthfulcompetitor,1 1-year-oldSteveNewman.Butallthreewere exploitedbyTESTERsincetheyallcalculatedthatthebestthingtodo 2.Unlikethefirstround,therearenoend-gameeffectsinthesecondround.Forthis reasonwenolongerhavetoworryaboutthepossibilityofminorfluctuationsin thescores duetotheseend-gameeffects. 392JOURNALOFCONFLICTRESOLUTION withaprogramthatcooperatedjustoverhalfthetimeafterone's owncooperationwastokeeponcooperating.Actuallytheywouldhave beenbetteroffdoingwhatTITFORTATandmanyotherhigh-ranking programsdid,whichistodefectimmediatelyonthesecondmovein responsetoTESTER'sdefectiononthefirstmove.Thiswouldhave elicitedTESTER'sapologyandthingswouldhavegonebetterthere- after. Theothernonnicerepresentative,TRANQUILIZER,illustratesa differentwayoftakingadvantageofmanyrules.TRANQUILIZER wassubmittedbyCraigFeathersandcamein27thinthetournament. Therulenormallycooperatesbutisreadytodefectiftheotherplayer defectstoooften.Thustherule tendstocooperateforthefirstdozenor twomovesiftheotherplayeriscooperating,butthenitthrowsina defection.Iftheotherplayercontinuestocooperate,thendefections becomemorefrequent.But as longas TRANQUILIZERis maintaining anaveragepayoffofatleast2.25pointspermove,itwillneverdefect twiceinsuccessionanditwillnotdefectmorethanone-quarterofthe time. AnexampleofhowarulecansufferfromTRANQUILIZERis providedbythesubmissionofGrahamEatherleywhichcamein14th. Eatherley'sruleis particularlyelegant.It keepstrackofhowmanytimes inthegametheotherplayerdefected.Aftertheotherplayerdefects,it defectswitha probabilityequaltothe ratioof theother'stotaldefections tothetotalmovestothatpoint.Eatherleyreportedthathewas fascinatedbytheresultsofthefirstroundandheimmediatelycoded mostoftheroundoneentriesandthesupplementaryrulesinorderto provideasuitable"provingground"forhisownroutines.Heevensent ina tournamentmatrixof20 rulesplayingeachothertoshowhowwell hisownselectedsubmissiondid.Heexperimentedwithmanyrulesand tookintoaccounthisexpectationthattherewouldbeamixof cooperativeandcompetitiverules.Thefactthatsomuchworkledto suchanelegantrulewasespeciallypleasing. Intheactualenvironmentofroundtwo,Eatherley'sruledidexactly aswellasTITFORTATwithfourofthefiverepresentatives.Butwith TRANQUILIZERitaveraged50.4pointsworsethatTITFORTAT did.Thereasonis thatbythetimeTRANQUILIZERstarteddefecting, it hadalreadyestablisheda goodrecordof cooperation.Thismeantthat Eatherley'srulewasunlikelytodefectafterthefirstoreventhesecond orthirddefectionofTRANQUILIZER.Sothedefectionscamemore Axelrod/PRISONER'SDILEMMACHOICE393 and more often, until an occasionalresponse from EATHERLEY was generated which in turn helped keep the proportion of defections under control.But since EATHERLEYwouldonly respond to a defection in proportion to theother's percentage of defection in the entire game so far, it usuallydidnotrespond. Table 5 shows the sequence of moves in one of the five games played between EATHERLEYand TRANQUILIZER.In this game TRAN- QUILIZER first defected on move 23, but EATHERLEY then had only a small chance of responding. It was not until after TRANQUILIZER's eighthdefectiononmove58thatEATHERLEYfirst defected.But TRANQUILIZERcontinued to throw in well-spaced defections which wereonlyoccasionallymetbyaresponseofadefectionfrom EATHERLEY.TheresultwasthatEATHERLEYwassuccessfully exploited,gettingonly387points,whileTRANQUILIZERgotan impressive 492pointsin this game. What it takes to do well with TESTER and TRANQUILIZERcan be put in a single word:provocability.A rule is provocable if it immediately defects after an "uncalled for" defection from the other. Nowjustwhat is an "uncalled for" defection is not very precise. But surely what TESTER and TRANQUILIZERdois uncalled for. The first one defects on the very firstmove,andtheseconddefectsafter adozenor twomutual cooperations.Iftheotherruleisnotprovokedtoanimmediate response, these rules (and many others like them) will simply take more and more advantage of such an easy-going player. So while it pays to be nice (by notbeing the first todefect), it also pays tobe provocable (by being ready to defect immediately after an uncalled for defection by the other). Howdoes this fit in with the observation from the first round of the tournament that it paid tobe forgiving of the other's defections? First, forgivenesswasdefinedinterms ofpropensity tocooperateafter the other player defected-andthis propensity was taken to be either long- term orshort-term propensity.3 Tobe provocableis tohaveashort- termpropensitynottoforgiveadefectionwhichisuncalledfor. Presumably it still pays tobe forgiving when you are in a rut of mutual defection,or asequence ofalternating exploitationfrom an echoof a singledefection.Butatleastin thesecondround where there were a number of rules which were lookingfor others whomight be tooeasy- going,italsopaid tobeprovocablesoasnottobeexploited. 3.ThisisabroaderdefinitionofforgivenessthantheoneusedbyRapoportand Chammah(1965:72-73),whichistheprobabilityofcooperationonthemoveafter receivingthesucker'spayoff,S. 394JOURNALOFCONFLICTRESOLUTION TABLE5 Illustrative Game BetweenEATHERLEYand TRANQUILIZER moves1-2011111111111111111111 moves21-4011212111211111111211 moves41-6011211111121111211232 moves61-8011121211211111112111 moves81-10021121111112311111112 moves101-12011111111112111211111 moves121-14023111121211111121111 moves141-15112111111111 Scoreinthisgame:EATHERLEY387,TRANQUILIZER492 Legend:1bothcooperated 2EATHERLEYonlycooperated 3TRANQUILIZERonlycooperated 4neithercooperated THETOPTHREERULES Niceness,forgiveness,andprovocabilityare all well-illustratedby the winningrule,TITFORTAT.It is neverthefirsttodefect,it forgivesan isolateddefectionaftera singleresponse,butit is alwaysprovokedbya defectionnomatterhowgoodtheinteractionhasbeensofar. TherulewhichcameinsecondwassubmittedbyDannyChampion ofLasCruces,NewMexico.Thisrulecooperatesonthefirst10 moves andplaystit-for-tatfor15moremoves.After25moves,theprogram cooperatesunlessallofthefollowingare true:theotherplayerdefected onthepreviousmove,theotherplayerhascooperatedlessthan60% of thetime,andtherandomnumberbetween0and1 isgreaterthanthe otherplayer'scooperationrate. Liketheothertop-rankingentries,thisrulewasniceandtherefore didjustaswellasTITFORTATwiththethreerepresentativeswhich werethemselvesnice.Champion'srulewasprovokedbyTRAN- QUILIZER'slatedefections,andsodidwellwithit.WithTESTER, however,Champion'sruledid12pointsworsethanTITFORTAT because,whileCHAMPIONwasgivingitsunconditionalcooperation onthefirst10 moves,itwasexploitedfivetimesbyTESTER.Butthis didnotgetoutofhand,becauseonceCHAMPIONdefectedonmove 11,TESTERapologizedandtheycooperatedthereafter. Axelrod/PRISONER'SDILEMMACHOICE395 ThethirdplacerulewassubmittedbyOttoBWrufsen ofUnder6y, Norway.Theruleplaystit-for-tatexceptthatithasacheckforecho effects,forRANDOM,andfora veryuncooperativeotherplayer.The checkforanechoeffectisactuallytooweak.It testsforthreemovesof alternatingexploitation,andifthisisfound,itsubstitutescooperation foritsnextdefectionnomatterhowmanymoveslaterthisdefection occurs.ThetroubleisthatwhenplayingTRANQUILIZER,TRAN- QUILIZERsometimesdefectsonmoven andn+2 therebysettingup an apparentechoeffect.ThenwheneverTRANQUILIZERnextdefects, Borufsen'srulewillnotbeprovoked.Thisunnecessarytypeof cooperationnotonlycoststwopointseachtimeithappensbutalso slightlyincreasesTRANQUILIZER'spropensitytodefectin the future. THEROLEOFTHEENVIRONMENT THELESSONSDRAWNFROMROUNDONE ThereportonthefirstroundofthePrisoner'sDilemmacomputer tournament(Axelrod,1980)concludedthatitpaidtobenotonlynice butalsoforgiving.Toillustratethatpoint,itwasshownthatsuch forgivingdecisionrulesasTITFORTWOTATSandREVISED DOWNINGwouldhavedoneevenbetterthanTITFORTATinthe environmentofthefirstround. Inthesecondround,manycontestantsapparentlyhopedthatthese conclusionswouldstillberelevant.Oftheentries,39werenice,and nearlyallofthemwereatleastsomewhatforgiving.TITFORTWO TATSitselfwassubmittedby an evolutionarybiologistfromtheUnited Kingdom,JohnMaynardSmith.Butitcameinonly24th.Aswehave seen,REVISEDDOWNINGwassubmittedtwice.Butinthesecond round,itwasinthebottomhalfofthetournament. Whatseemstohavehappenedisaninterestinginteractionbetween peoplewhodrewonelessonandpeoplewhodrewanotherfromthe first round.LessonOnewas"beniceandforgiving."LessonTwowas"if othersare goingtobe nice andforgiving,it pays totry to take advantageof them."ThepeoplewhodrewLessonOnesufferedinthesecondround fromthosewhodrewLessonTwo.Aswehaveseen,ruleslike TRANQUILIZERandTESTERwereeffectiveatexploitingrules whichweretooeasy-going.ButthepeoplewhodrewLessonTwodid notthemselvesdoverywelleither.Thereasonis thatin tryingtoexploit 396JOURNALOFCONFLICTRESOLUTION otherrules,theyofteneventuallygotpunishedenoughtomakethe wholegamelessrewardingforbothplayersthanpuremutualcoopera- tionwouldhavebeen.Forexample,TRANQUILIZERandTESTER themselvescameinonly27thand 46thplace,respectively.Theyeachdid betterthanTITFORTATwithfewerthanone-thirdoftherules.None oftheotherentrieswhichtriedtoapplyLessonTworankedatthetop either. WhiletheuseofLessonTwotendedtoinvalidateLessonOne,noone wasabletobenefitmorethantheywerehurtinthetournamentbyhis attempttoexploittheeasy-goingrules.Themostsuccessfulentries tendedtoberelativelysmallvariationsonTITFORTATwhichwere designedtorecognizeandgiveup on a seeminglyRANDOMplayeror a veryuncooperativeplayer.Buteventheimplementationsoftheseideas didnotdobetterthanthepureformofTITFORTAT.SoTITFOR TAT,whichgotalongwithalmosteveryone,wonthesecondroundof thetournamentjustasithadwonthefirstround. ROBUSTNESSOFTHETOURNAMENTRESULTS Wouldtheresultsofthesecondroundhavebeenmuchdifferentif the distributionofentrieshadbeensubstantiallydifferent?Putanother way,howrobustaretheresults? Agoodwaytoexaminethisquestionistoconstructaseriesof hypotheticaltournamentseachwitha differentrelativeweightingof the typesofrulesparticipating.Thefiverepresentativescanserveusagain, sinceeachcanbethoughtofashavingalargeconstituency.Together withtheunrepresentedconstituencyofthe residuals, thesefivecon- stituenciesfullyaccountfortheperformanceofeachruleinthe tournament.Thisallowsustoseewhatwouldhavehappenedif oneof theconstituencieshadbeenmuchlargerthanitactuallywas.Tobe specific,wewillconstructthehypotheticaltournamentswhichwould haveresultedifagivenconstituencyhadbeenfivetimesaslargeasit actuallywas.Sincetherearesixconstituencies,thisgivesussix hypotheticaltournaments.Eachofthesehypotheticaltournaments representsa substantialvariationontheoriginaltournamentbecauseit quintuplesthesizeofoneoranotherofthesixconstituencies.Andeach representsa differentkindof variationsinceeachis basedonmagnifying theeffectofagivenaspectofthetournament.4 4.Hereishowthehypotheticaltournamentscoresarecalculated.Tomakethe constituencyofagivenrepresentative fivetimesas large asit actuallywas, letT'=T + 4cs Axelrod/PRISONER'SDILEMMACHOICE397 Infact,thescoresinthesehypotheticaltournamentscorrelatefairly wellwiththescoresintheoriginaltournament.If theresidualswerefive timesaslargeastheyactuallywere,thetournamentscoreswouldstill havea correlationof.82 withthescoresin the actualtournament.Andif theconstituencyofanyofthefiverepresentativesweremadefivetimes aslargeasitactuallywas,thetournamentscoreswouldstillbe correlatedfrom.90to.96withthetournamentscoresoftheactual secondround.Thismeansthattheoverallresultswouldhavebeenfairly stableevenifthedistributionofentriesbytypesofprogramhadbeen quitedifferentfromwhatit actuallywas.Thustheoverallresultsofthe secondroundarequiterobust. Butmovingfromthetournamentasawholetotheidentityofthe winner,wecanalsoaskhowTITFORTATwouldhavedonein these sixhypotheticaltournaments.Theansweristhatitwouldstillhave comeinfirstplaceinfiveofthesixhypotheticaltournaments.Thisis a verystrongresultsinceitshowsthatTITFORTATwouldstillhave beenthebestruleofthosesubmittedunderverywidevariationsinthe environmentithadtoface. TheoneexceptiontoTITFORTAT'ssuccessinthehypothetical tournamentsisaveryinterestingone.Hadtheconstituencyofthe REVISEDSTATETRANSITIONrulebeenfivetimesaslargeasit actuallywas,TITFORTATwouldhavecomeinsecond.Firstplace wouldhavebeenwonbyarulewhichrankedonly49thintheactual tournament.ThsrulewassubmittedbyRobertLeylandofAuckland, NewZealand.Itsmotivationis similartoTRANQUILIZER'sin thatit startsoffcooperativelybutthenseeshowmuchit canget awaywith.As canbeseenfromTable4,Leyland'srulecamein49thlargelybecauseit didsopoorlywiththethirdrepresentativeandwithTRANQUILIZER. Butitdiddo90pointsbetterthanTITFORTATwithREVISED STATETRANSITION,sincethatrulewasquitewelltakeninbythe earlycooperations.IftheconstituencyoftheREVISEDSTATE TRANSITIONrepresentativehadbeenfivetimesaslargeas it actually was,Leyland'srule wouldactuallyhavedonebetterthanTITFORTAT oranyothersubmittedruleinthetournamentasawhole. whereT'isthenewtournamentscore.Tistheoriginaltournamentscore,cisthe coefficient in the regression equation of the representative whose effect is to be magnified, and s is the score of the given rule with that representative. It should be noted that the idea of a "constituency" of a representative is defined in this way, and that a typical rule is part of the constituency of several representatives. The hypothetical tournament in which the residuals are given added weight is constructed in an analogousmanner with T' - T + 4r, where r is theresidual in theregression equationfor thescore ofa given rule. 398JOURNALOFCONFLICTRESOLUTION ThefactthatTITFORTATwonfiveofthesixmajorvariantsof the tournamentandcameinsecondin thesixth,showsjusthowrobustTIT FORTAT'svictoryreallywas. SURVIVALOFTHEFITTEST THEECOLOGICALPERSPECTIVE Anotherwaytoexaminetherobustnessof theresultsis toconstructa wholesequenceofhypotheticalroundsofthetournament.Someofthe rulesaresounsuccessfulthattheyare unlikelytobe triedagainin future tournaments,whileothersaresuccessfulenoughsothattheircontinued presenceinlatertournamentswouldbemorelikely.Forthisreason,it wouldbehelpfultoanalyzewhatwouldhappenoveraseriesof tournamentsifthemoresuccessfulrulesbecamealargerpartofthe environmentforeachrule,andthelesssuccessfulrulesweremetless often. Evolutionarybiologyprovidesausefulwaytothinkaboutthis problem(Trivers,1971;Dawkins,1976:197-202;MaynardSmith,1978). Imaginethatthereare a verylargenumberofanimalsofasinglespecies, whichinteractwitheachotherquiteoften.Supposetheinteractionstake theformofaPrisoner'sDilemma:Whentwoanimals meet,theycan cooperatewitheachother,notcooperatewitheachother,or oneanimal couldexploittheother.Supposefurtherthateachanimalcanrecognize individualsithasalreadyinteractedwithandcanremembersalient aspectsoftheirinteractionsuchaswhethertheotherhas usually cooperated.Thetournamentcanthenberegardedasasimulationofa singlegenerationofsuchanimalswitheachdecisionrulebeing employedbylargeandequalnumbersofindividuals.Oneconvenient implicationofthisinterpretationisthatagivenanimalis justaboutas likelytointeractwithananimalusingitsowndecisionrule as it is torun intoananimalwithanyotherrule. Theinterestingpartofthisanalogyisthatitallowsustosimulate futuregenerationsofatournament.Wesimplyhavetointerpretthe averagepayoffreceivedbyanindividualasproportionaltothat individual'sexpectednumberofoffspring.Forexample,ifonerule gets twiceashighatournamentscoreintheinitialroundasanotherrule, thenit willbe twiceas well-representedin thenextround.Thiscreatesa simulatedsecondgenerationofthetournamentinwhichtheaverage Axelrod/PRISONER'SDILEMMACHOICE399 score achieved by a rule is the weighted average of its score with each of the rules, where the weights are proportional to the success of the other rules in the initial generation. Thus RANDOM,for example, will be less important in the second generation while TIT FOR TAT and the other high-ranking rules will bemore important. THEDYNAMICSOFTHEECOLOGY Thesimulationofthisprocess forthePrisoner's Dilemmatourna- ment is actually quite straightforward. The tournament matrix (which is displayed in abbreviated form in Table 3) provides the expected payoff when an individual of one type interacts with an individual of another type. Starting with proportions of each type in a given generation, it is only necessary tocalculate the proportions which will exist in the next generation.Thisisdonebycalculatingtheweightedaverageofthe scoresofagivenrule withallotherrules where theweightsare the numbers oftheother rules which existin thecurrent generation. The numbersofagivenruleinthenextgenerationisthentakentobe proportional to the product of its numbers in the current generation and its score in thecurrent generation. The results provide an interesting story. The first thing that happens isthat the lowestranking11 entries fall tohalf their initial size by the fifthgenerationwhilethemiddle-ranking entriestendtoholdtheir ownandthetop-rankingentriesslowlygrowinsize.Bythe50th generation,the rules which ranked in thebottomthird of the tourna- ment have virtually disappeared, while most of those in the middle third have started to shrink, and those in the top third are continuing to grow (seeFigure1). Thissimulatessurvival ofthe fittest.Arule which issuccessfulon averagewiththecurrent distributionofrules inthepopulationwill become an even larger proportion of the environment of the other rules in the next generation. At first a rule which is successful with all sorts of ruleswillproliferate,butlaterastheunsuccessfulrulesdisappear, success requires goodperformance withother successful rules. Thisis anecologicalperspective rather than astrictly evolutionary perspective. There are no new rules of behavior introduced so there is no mutationtodrive evolution.5Butthere isachangingdistributionof given types of rules. The less successful rules become less common and the more successful rules proliferate. The statistical distribution of types 5.Foranevolutionaryperspective on the iterated Prisoner's Dilemma,see Axelrod (1979). 400JOURNALOFCONFLICTRESOLUTION TIT FORTAT1 .1403 //~~~~~2 .120 -,9 Z .1001 0* -J .080 CL 0 z 10 o.060 0 n- 0 Q.040 l-. .020 185 Oth 131412,15I 0200400600800 1000 GENERATIONS Figure1:SimulatedEcologicalSuccessoftheDecisionRules ofindividualschangesineachgeneration,andthischanges the environment with which eachoftheindividual typeshas to interact. Theprocess ofchange slowsas the remaining rules tend to achieve averages which are similar toeach other. By the 500th generation only Axelrod/PRISONER'SDILEMMACHOICE401 1 1rulesarelargerthantheirinitialsizeinthepopulation,andthese happentobetheruleswhichoriginallyrankedinthetop1. Together these11 rulesformed96% ofthepopulationat thattime.Therule which rankedfifthin thetournamentgrewtothreetimesitsoriginalsizein the populationandthenbegantosinkaftergeneration100.Therulewhich rankedeighth,andwastheonlynonnicerulein thetop15, grewtofour timesitsoriginalsizebutbegantoshrinkaftergeneration150 toreach onlyathirdofitsoriginalsizebythe1000thgeneration. Theresultsshowthattheentrieswhichrankedatthetopinthe tournamentactuallydoquitewellwithothersuccessfulentries,andthus didnotrelyfortheirsuccessmerelyontheirabilitytogethighscores withunsuccessfulrules.Whiletherearea fewexceptions,suchas rules5 and8, mostof the top-rankingentriescontinuedtodoquitewellfor very longperiods. TheresultsalsoprovideyetanothervictoryforTITFORTAT.TIT FORTAThada veryslightleadin theoriginaltournament,andnever lostthisleadinsimulatedgenerations.Bythe1000thgenerationit was 14.5% ofthewholepopulation,followedby the thirdplacerule at13.9% andthenthesecondplacerueat13.1%.AndTITFORTATwasstill growingat.05%pergenerationwhichwasafasterratethananyother rule. ISTITFORTATTHEBESTDECISIONRULE? Wehaveseenthatinthesecondround,TITFORTATachievedthe highestaveragescoreofthe63rulesin thetournament.It alsoachieved thehighestscorein fiveofthesixhypotheticaltournamentswhichwere constructedbymagnifyingtheeffectsofdifferenttypesofrules.Andin thesixthhypotheticaltournamentit cameinsecond.Finally,TITFOR TATneverlostitsfirst-placestandinginasimulationoffuture generationsofthetournament. Addedtoits victoryin the first roundof thetournament,and its fairly goodperformancewithhumansubjects(Oskamp,1971; Wilson,1971), onemightwellaskwhetherTITFORTATisthebestdecisionrulefor thePrisoner'sDilemma.Ithinknot,andherearemythreereasons. First,TITFORTATcouldhavebeenbeatenin thesecondroundby anyrulewhichwasabletoidentifyandnevercooperatewithRAN- 402JOURNALOFCONFLICTRESOLUTION DOM,whilenotmistakingotherrulesforRANDOM.Sucharule wouldbehardtodesign,butpresumablyitwouldbepossible.6 Second,hadonlytheentrieswhichactuallyrankedinthetophalf beenpresent,thenTITFORTATwouldhavecomein fourthafterthe oneswhichactuallycamein25th,16th,and8th.7 Whilethesimulation offuturetournamentsisamoremeaningfulwayofalteringthe environmenttotakeaccountof thedifferentialsurvivalof thefittest,the factremainsthatinatournamentamongjustthetophalf,TITFOR TATcomesinonlyfourth. Third,themostpowerfulargumentagainstabsolutesuperiorityof TITFORTATis theonedemonstratedat thestartof thisarticle:There isnobestruleindependentoftheenvironment. Sincethereisnoabsolutelybestrule,whatwecansayforTITFOR TAT'ssuccessesisthatitisaveryrobustrule;itdoesverywellovera widerangeofenvironments.Partofitssuccessmightbe thatotherrules anticipateitspresenceandaredesignedtodowellwithit.Doingwell withTITFORTATrequirescooperatingwithit,andthisinturnhelps TITFORTAT.Aswehaveseen,evenruleslikeTESTERwhichwere designedtoseewhattheycangetawaywith,quicklyapologizetoTIT FORTAT.AnyrulewhichtriestotakeadvantageofTITFORTAT willsimplyhurtitself.Sincethisissowell-understood,andthe possibilityofmeetingTITFORTATis sosalient,and TITFORTATis soeasytorecognize,itreapsthebenefitsofitsownnonexploitability. Ontheotherhand,TITFORTATforegoesthepossibilityof exploitingotherrules.Whilesuchexploitationisoccasionallyfruitful, thereareseriousproblemsintryingtoturnitintoaneffectivestrategy overawiderangeofenvironments.Wehaveseen,forexample,thatif exploitableruleslikeREVISEDSTATETRANSITIONwereamuch largerpartofthetournamentthantheyactuallywere,thenanotherwise low-rankingrulewouldhavebeenabletowinthetournament.Butthe problemswithtryingtoexploitothersaremanifold.In thefirstplace,if 6.Evenknowingtheruleswhichweresubmitteddoesnotmakethetaskeasy.I triedto designsucha rule.ItplayedTITFORTATexceptthatitwouldalwaysdefectaftermove 100 if itsscorethenwaslessthan2.75permove.Thisruleaveraged431.3pointswiththe63 rulesinthetournament,whichis ascoreexceededbythreeoftherulesthatwereactually submitted.Theproblemwasthattherewerestillfourruleswhichthisonegaveupon,but whichTITFORTATdidbetterwith. 7.ThesethreerulesdowellwiththesameonesasdoTESTERandTRANQUILI- ZER,andforthesamereasons.Notbeingnice,they are abletoexploitruleswhicharetoo forgivingfortheirowngood.Butinthetournamentasawholetheserulesdidnotdoas wellas TITFORTATbecauseofthefrequencyofless-forgivingrulesinthebottomhalfof theoverallranking. Axelrod/PRISONER'SDILEMMACHOICE403 a rule defectstosee whatit can get awaywith,it risksretaliationfromthe ruleswhichareprovocable.Inthesecondplace,oncemutualrecrimi- nationssetin,itcanbedifficulttoextractoneselffromthem.And, finally,theattempttoidentifyandgiveuponunresponsiverules(such asRANDOMorexcessivelyuncooperativerules)seemstohaveoften mistakenlyledtogivinguponruleswhichwerein factsalvageablebya morepatientrulelikeTITFORTAT.Beingabletoexploitthe exploitablewithoutpayingtoohigha costwiththeothersis a taskwhich wasnotsuccessfullyaccomplishedbyanyoftheentriesin roundtwoof thetournament. WhataccountsforTITFORTAT'srobustsuccessis its combination ofniceness,provocability,andforgiveness.Itsnicenesspreventsit fromgettingintounnecessarytrouble.Its provocabilitydiscouragesthe othersidefrompersistingwheneverdefectionistried.Anditsforgive- nesshelpsrestoremutualcooperation. REFERENCES AXELROD,R.(1980)"Effective ChoiceinthePrisoner'sDilemma."J.ofConflict Resolution24: 3-25. ---(1979)"The EvolutionofCooperationin thePrisoner's Dilemma." Institute of PublicPolicyStudies,University ofMichigan,DiscussionPaper143. DAWKINS,R.(1977)The SelfishGene. NewYork: OxfordUniv.Press. MAYNARDSMITH,J.(1978) "The Evolution of Behavior." Scientific Amer. 239: 176- 192. OSKAMP,S. (1971) "Effects of Programmed Strategies on Cooperation in the Prisoner's Dilemmaand Other Mixed-MotiveGames." J.ofConflictResolution15: 225-259. RAPOPORT,A. and A. M. CHAMMAH(1965) Prisoner's Dilemma. Ann Arbor: Univ. ofMichigan Press. TRIVERS,R.T.(1971)"The EvolutionofReciprocalAltruism." Q.Rev.ofBiology 46: 35-57. WILSON,W.(1971)"Reciprocation andOther TechniquesforInducing Cooperation inthePrisoner's DilemmaGame." J.ofConflictResolution15: 167-195.