Evolving New StrategiesEvolving New Strategies

The Evolution of Strategies in the The Evolution of Strategies in the Iterated Prisoner’s DilemmaIterated Prisoner’s Dilemma

01 / 25

What is the Prisoner’s Dilemma?What is the Prisoner’s Dilemma?

There are two prisonersThere are two prisoners Each one has taken part in the same criminal actEach one has taken part in the same criminal act The authorities are interrogating each oneThe authorities are interrogating each one Each prisoner can choose to keep their mouth shut or rat out their Each prisoner can choose to keep their mouth shut or rat out their

partnerpartner If both prisoners stay quiet, they each get n months of jail timeIf both prisoners stay quiet, they each get n months of jail time If only one prisoner gets ratted out, that prisoner gets n + x months If only one prisoner gets ratted out, that prisoner gets n + x months

of jail time while the other prisoner gets n – y months of jail timeof jail time while the other prisoner gets n – y months of jail time If the prisoners rat each other out, they each get n + z months of jail If the prisoners rat each other out, they each get n + z months of jail

time.time. In this case, n, x, y, and z are all greater than zero.In this case, n, x, y, and z are all greater than zero. In this case, x is greater than z.In this case, x is greater than z.

02 / 25

What is the Iterated Prisoner’s What is the Iterated Prisoner’s Dilemma?Dilemma?

Prisoner’s Dilemma performed several timesPrisoner’s Dilemma performed several times The two criminals have committed several The two criminals have committed several

crimes togethercrimes together They are interrogated for each crime, with each They are interrogated for each crime, with each

set of interrogations being an instance of the set of interrogations being an instance of the original Prisoner’s Dilemmaoriginal Prisoner’s Dilemma

These interrogations are performed in sequence These interrogations are performed in sequence (or (or iterativelyiteratively), and the jail time distributed to ), and the jail time distributed to each prisoner is cumulativeeach prisoner is cumulative

03 / 25

How does the IPD relate to GAs?How does the IPD relate to GAs?

No optimal solutionNo optimal solutionNo real strategyNo real strategyNo clueNo clueHard problemHard problem

So back to the paperSo back to the paper

04 / 25

What This Paper ShowsWhat This Paper Shows

GAs in a rich social settingGAs in a rich social settingAdvantage of developing new strategiesAdvantage of developing new strategies

One parentOne parentTwo parentTwo parent

Early commitments to pathsEarly commitments to pathsEvolutionary processes optimal or arbitraryEvolutionary processes optimal or arbitrary

05 / 25

How Does It Show It?How Does It Show It?

SimulationSimulationMultiple casesMultiple casesComparative outputComparative output

06 / 25

The SimulationThe Simulation

Specify the environmentSpecify the environmentSpecify the encodingSpecify the encodingTesting the effects of random mutationTesting the effects of random mutationRun the simulationRun the simulationAnalyze the resultsAnalyze the results

07 / 25

The EnvironmentThe Environment

Prisoner’s dilemmaPrisoner’s dilemmaMultiple prisonersMultiple prisonersGoal is to achieve mutual cooperationGoal is to achieve mutual cooperation Individuals may meet more than onceIndividuals may meet more than once

08 / 25

Initial ExperimentInitial Experiment

Original strategies were submitted by Original strategies were submitted by fourteen peoplefourteen peopleGame TheoryGame TheoryEconomicsEconomicsSociologySociologyPolitical SciencePolitical ScienceMathematicsMathematics

Various levels of intricacyVarious levels of intricacy

09 / 25

Initial ExperimentInitial Experiment

Most complex strategyMost complex strategyMarkov process Markov process Bayesian inferenceBayesian inference

Least complex strategyLeast complex strategyTFTTFT

TFT won TFT won

10 / 25

Second ExperimentSecond Experiment

Sixty-two entriesSixty-two entriesSix countriesSix countriesComputer hobbyists, professorsComputer hobbyists, professors

TFT was submitted againTFT was submitted again It wonIt won

11 / 25

The GAThe GA

PopulationPopulationEncodingEncodingGenerationGenerationCrossoverCrossoverMutationMutationFifty GenerationsFifty Generations

12 / 25

PopulationPopulation

Twenty chromosomesTwenty chromosomesSeventy genesSeventy genes

13 / 25

EncodingEncoding

For each prisoner’s dilemma, there are For each prisoner’s dilemma, there are four possibilitiesfour possibilities

Each “player” has memoryEach “player” has memoryWhat each gene representsWhat each gene represents

14 / 25

A Single GenerationA Single Generation

Multiple gamesMultiple games Each game had one-hundred and fifty-one movesEach game had one-hundred and fifty-one moves Each chromosome played eight othersEach chromosome played eight others

Fitness was assignedFitness was assigned Ratted out – Zero pointsRatted out – Zero points Mutually ratted out – One pointMutually ratted out – One point Mutual cooperation – Three pointsMutual cooperation – Three points You ratted, other person stayed quit – Five pointsYou ratted, other person stayed quit – Five points

15 / 25

CrossoverCrossover

Fitness proportional selectionFitness proportional selection Involved standard deviation from meanInvolved standard deviation from mean

Strictly ten crossoversStrictly ten crossoversSingle pointSingle pointTwo parentsTwo parents

16 / 25

MutationMutation

Single gene flipSingle gene flipOne gene per two chromosomesOne gene per two chromosomes

17 / 25

ResultsResults

Median resultant memberMedian resultant memberJust as good as TFTJust as good as TFTResembled TFTResembled TFT

Five properties were foundFive properties were found Don’t rock the boatDon’t rock the boat Be provocableBe provocable Accept apologiesAccept apologies ForgetForget Accept a rutAccept a rut

18 / 25

ResultsResults

ADJUSTERADJUSTERSpecial chromosome which consistently Special chromosome which consistently

seeks to exploitseeks to exploitTFTTFTMajority of other chromosomesMajority of other chromosomes

19 / 25

ResultsResults

Twenty-five percent of runsTwenty-five percent of runsMedian was betterMedian was betterExploit one chromosomeExploit one chromosome

20 / 25

Results: Why is this important?Results: Why is this important?

Chromosomes had to learnChromosomes had to learnDiscriminatory based on evidenceDiscriminatory based on evidenceSelf adjusting for exploitationSelf adjusting for exploitationNo alienationNo alienation

Break primary rule of first tournamentBreak primary rule of first tournamentBe nice? I don’t think soBe nice? I don’t think so

21 / 25

Results: Misleading?Results: Misleading?

MedianMedianExploitativeExploitativeMenacingMenacingA true criminal?A true criminal?

Fixed size population and tournamentsFixed size population and tournamentsSimulate real evolutionSimulate real evolution

22 / 25

Results: A Slight TwistResults: A Slight Twist

Asexual reproductionAsexual reproductionTFTTFTLess than half of the mediansLess than half of the medians

Changing environmentChanging environmentPlay against everyonePlay against everyoneEveryone starts aggressiveEveryone starts aggressive

Fitness rapidly declinesFitness rapidly declinesFitness begins to even outFitness begins to even outFitness begins to riseFitness begins to rise

23 / 25

ConclusionsConclusions

The GA is good for searching, large, multi-The GA is good for searching, large, multi-dimensional spacesdimensional spaces

Multiple parent crossover helpsMultiple parent crossover helpsArbitrary aspects of evolutionArbitrary aspects of evolution

Hitch hikersHitch hikersExploration vs. ExploitationExploration vs. Exploitation

Selection PressureSelection PressureEvolutionary Commitments can be Evolutionary Commitments can be

irreversibleirreversible

24 / 25

Related TopicsRelated Topics

MutationMutationCrossoverCrossover InversionInversionCoding principlesCoding principlesDominant/RecessiveDominant/RecessiveRate of evolutionRate of evolutionPopulation viscosityPopulation viscositySpeciation and nichesSpeciation and niches

25 / 25