Running Axelrod's Tournament Again, With Noise This Time

Fifteen Programs Walk Into a Round Robin

We ran the tournament. Fifteen strategies, every pair, four hundred times over, at ten different levels of static. Four hundred and eighty thousand matches in the main sweep, ninety-six million rounds of the prisoner's dilemma, and a leaderboard that would not sit still.

Tit for tat did not win.

It did not lose either. At zero noise it came third, by a margin so small that two of the three top places are a statistical dead heat. Then we turned the noise on and the board rearranged itself, twice, and by the time one move in five was landing wrong the winner was always defect.

That is the whole story in five numbers, and every one of them came out of a program the club wrote and ran on a laptop. Nothing below was observed. Nobody played anything. The computation is the experiment, and the rest of this article is about how far a computation like that can be pushed before it starts saying things it has no right to say.

What We Actually Built

The stage game is Axelrod's, with his numbers. Defecting against a cooperator pays \(T = 5\), mutual cooperation pays \(R = 3\), mutual defection pays \(P = 1\), and cooperating into a defector pays \(S = 0\). A match is 200 rounds, the length of his first tournament [1]. He ran a second one the same year with 62 entries, and tit for tat won that as well [2]. A round robin is every unordered pair of strategies playing one match, including each strategy against a fresh copy of itself, which is how Axelrod scored his entrants against their own twins. A strategy's tournament score is its mean payoff per round averaged over all fifteen opponents.

Noise is an execution error. Each player's intended move on each round is flipped with independent probability \(z\), and the flipped move is what both players see. This is the trembling hand, sometimes called misimplementation noise, and it is not the same thing as misperception, where only the victim of the error knows about it. The difference matters and we come back to it.

The library holds fifteen classical strategies. Always cooperate and always defect are the two poles. Random flips a fair coin every round. Tit for tat opens cooperatively then copies. Generous tit for tat is tit for tat that forgives a defection with probability \(g = 1/3\), which is the level Molander derived for exactly this payoff matrix [4]. Tit for two tats waits for two defections before it reacts. Suspicious tit for tat opens with a defection. Grim trigger cooperates until it is crossed once and then defects forever. Pavlov repeats its last move after a good payoff and switches after a bad one, a rule Nowak and Sigmund argued outperforms tit for tat under noise [8][9]. Contrite tit for tat tracks who is in good standing, so that after its own accidental defection it accepts one retaliation instead of echoing it [5]. Joss is tit for tat that sneaks in an extra defection one time in ten. Tester opens with a defection to see what happens. Alternator cooperates and defects on alternate rounds. Hard tit for tat defects if the opponent defected on any of the last three rounds. Prober opens with D, C, C and then exploits anyone who let it get away with that.

The evolutionary half takes the measured payoff matrix and runs discrete replicator dynamics on it [11]. Every strategy starts at frequency \(1/15\), and each generation

Strategies that earn more than the population average grow. The rest shrink. Four thousand generations, with no mutation and no spatial structure.

Before Believing Any of It

A tournament program is easy to write and easy to get subtly wrong, so the run checks itself before it reports anything. Four checks, printed in full in the raw output.

The payoff structure. \(T > R > P > S\) holds as \(5 > 3 > 1 > 0\). The second inequality, \(2R > T + S\), holds as \(6 > 5\). Without that one, two players taking turns to exploit each other would out-earn two players cooperating, and the game would not be a prisoner's dilemma at all.

Twenty-five pairings solved by hand. At zero noise most of these strategies are deterministic, so the payoff of a match can be worked out with a pencil. Tit for tat against always defect gets cooperated on once and then locks into mutual defection, which is \((S + 199P)/200 = 0.995\) exactly. Alternator against tit for tat gets \(R\) on round one and then takes \(T\) on a hundred rounds and \(S\) on ninety-nine, which is \((3 + 500)/200 = 2.515\). Prober against always cooperate probes, finds a pushover and exploits it for 197 rounds, which is \((5 + 6 + 985)/200 = 4.980\). Fifty such values, counting both sides of each pairing, and all fifty came back identical to the closed form to every printed digit.

Zero noise means zero variance where it should. With the noise off, a match between two deterministic strategies has nothing random in it, so its payoff must be the same in all four hundred repetitions. Across the 144 deterministic cells of the matrix the largest spread was exactly 0.000. Across the 81 cells involving a coin the spread ran up to 2.41, which is the other half of the same check.

The replicator dynamics conserve population. The frequencies are never renormalised after the update, so if the arithmetic is right the total must stay at one on its own. Over four thousand generations at every one of ten noise levels, the largest deviation of \(\sum_i x_i\) from unity was \(5.551 \times 10^{-16}\), or two and a half units in the last binary place.

There is a fifth check, and it is the one we are most pleased with. Two tit-for-tats playing each other under noise is a four-state Markov chain on the pair of moves actually played, which can be solved exactly for the same 200-round horizon without simulating anything. We solved it separately and compared it with the simulated diagonal cell at all ten noise levels. The largest disagreement anywhere was 1.84 standard errors, which is about what the largest of ten standard normals should be.

Zero Noise, and the First Upset

Here is the noiseless leaderboard. We expected a formality. It was not one.

Generous tit for tat, 2.63999. Contrite tit for tat, 2.62932. Tit for tat, 2.62890. Then a gap of more than a sixth of a point down to Pavlov in fourth, and after that the field falls away.

The top three are separated by eleven thousandths of a point per round, so the honest question is whether any of it is real. Because every strategy meets the same field in the same repetition, the sharp test is a paired difference rather than two separate error bars. Contrite tit for tat beats tit for tat by 0.00042 per round with a paired standard error of 0.00090, a \(t\) of 0.47, which is nothing. Those two are tied and we report them as tied. Generous tit for tat beats tit for tat by 0.01109 with a paired standard error of 0.00365, a \(t\) of 3.04. That one is not nothing. Three standard errors is a real gap, and it does not shrink when we throw more repetitions at it.

So the 1980 headline half survives. Tit for tat ties for the top of this field with contrite tit for tat, and sits three standard errors behind a strategy Axelrod never received. We are not going to pretend otherwise, and we are not going to quietly drop an entrant until the expected answer comes back.

What costs tit for tat the outright win is one opponent. Drop each strategy in turn, rerun the noiseless round robin over the remaining fourteen, and the culprit is obvious. Remove Joss and generous tit for tat's score collapses from first place to a dead heat: the winner becomes contrite tit for tat by 0.00054 per round over tit for tat, a difference our error bars cannot see. Everywhere else in the matrix, tit for tat and generous tit for tat score identically. Against Joss, tit for tat earns 1.215 and generous tit for tat earns 2.558.

Joss cooperates like tit for tat and then defects at random one tenth of the time. Tit for tat echoes the defection. Joss echoes the echo. The two of them fall into a feud that neither one is trying to have and neither one can end, and they both come out near the mutual-defection payoff. Generous tit for tat lets a defection go one time in three, which is enough to break the loop. The forgiveness costs it nothing against anybody else and buys it 1.34 points a round against Joss, and 1.34 divided by fifteen opponents is 0.089, which more than covers its winning margin. That is the entire result.

Working Notes From the Club Table

Turning the Noise Up

Now the interesting part. Ten noise levels from zero to one move in five, a full 400-repetition round robin at each, every level drawing its own independent stream from the master seed.

There are two handovers. Generous tit for tat holds the lead from zero up to 2% noise. At 5% it loses to contrite tit for tat, which then holds the lead through 10% and 15%. At 20% the board turns over completely and always defect takes it, with grim trigger second and hard tit for tat third. The three top strategies at that level are the three least forgiving ones in the library.

Tit for tat's own trajectory is the quiet story. Rank 3 at zero noise, rank 3 at 0.5%, rank 4 at 1%, rank 5 at 2% and 5%, and it never recovers. Its score at 5% noise is 2.18705 against the leader's 2.26490. The gap is 0.07785 per round with a paired \(t\) of 24.1 on 399 degrees of freedom, which is not a close thing. Over a 200-round match that is 15.57 points, and over the fifteen matches of a tournament it is 233 points, close to half of Axelrod's entire winning total.

Notice the two columns disagreeing with each other. The round robin says always defect wins at 20% noise. The evolutionary run at the same noise level says contrite tit for tat takes the whole population. Both are computed from the same payoff matrix, and neither is wrong. A round robin asks which strategy does best against a fixed, evenly weighted field including a lot of exploitable opponents. Replicator dynamics ask which strategy survives when the field stops being fixed and the exploitable opponents die out. Always defect is superb at eating a room full of cooperators and useless once it has finished eating them.

This split is the result Nowak and Sigmund built their heterogeneous-population argument on [7], and it is the reason a tournament ranking and an evolutionary outcome should never be quoted as if they were the same measurement.

The Arithmetic of One Slip

Take two tit-for-tats. Both cooperating, both happy, 3 points a round each. Now flip one move.

Player A intended C and played D. Next round B copies the D it saw, and A copies the C it saw, so B defects and A cooperates. The round after that they swap. From that point on they alternate, one of them taking \(T = 5\) while the other takes \(S = 0\), and each of them averages \((5+0)/2 = 2.5\) a round instead of 3. One slip, half a point a round, forever.

Now count the slips. A 200-round match gives 400 chances to err, two players times two hundred rounds. At \(z = 0.001\) the chance of getting through clean is \(0.999^{400} = 0.6702\), so exactly a third of matches go wrong at all. If a match does go wrong, the error lands somewhere uniformly along it, so on average half the match is spent in the alternating state, costing 0.25 a round for that match. Multiply: \(0.3298 \times 0.25 = 0.0825\) points a round expected loss.

The exact chain says 0.0985. Our envelope is 16% low because it ignores matches with more than one error, and a second error is not symmetric in its effects. From the alternating state, an error by the player who meant to cooperate produces mutual defection, which is absorbing until the next error. An error by the player who meant to defect produces mutual cooperation, which is a full repair. Both happen at the same rate, so the pair spends its time drifting among three regimes and the average sinks below the one-error estimate.

Push the noise all the way up and the arithmetic gets blunt. At \(z = 0.20\) the exact long-run payoff for two tit-for-tats is 2.2568. The average of all four cells of the payoff matrix, \((5+3+1+0)/4\), is 2.25. Two tit-for-tats at one-in-five error are, to three decimal places, two random number generators. Every bit of structure the strategy had has been eaten.

Letting Them Breed

The round robin is a single fixed room. The evolutionary run lets the room change. Start all fifteen strategies at equal frequency, feed them the measured payoff matrix, and let the ones earning above average grow.

At zero noise the population never resolves. Eight strategies survive to generation four thousand and the run is still creeping: contrite tit for tat at 0.2252, tit for tat at 0.2251, generous tit for tat at 0.2082, Pavlov at 0.1040, tit for two tats at 0.0695, grim trigger at 0.0604, hard tit for tat at 0.0584, always cooperate at 0.0493. Every one of them scores exactly 3 against every other one of them, so once the exploiters have been eliminated there is nothing left to select on. Mean payoff 3.0000. The population has reached the best possible outcome and has no way to choose among the strategies that got it there.

Add a tenth of a per cent of noise and that tie breaks instantly. At \(z = 0.001\) only contrite tit for tat and generous tit for tat survive, at 0.735 and 0.265. At \(z = 0.002\) contrite tit for tat holds 0.962. From \(z = 0.005\) upward it takes the entire population, at every noise level we tested, right through 20%.

That is a clean result and it is exactly what Wu and Axelrod predicted when they ran a noisy version of the same tournament thirty years ago [10]. Their finding was that generosity works best against players who have not adapted to noise, while contrition works best at restoring cooperation among players who have. They also reported that Pavlov is not reliable. Our two halves say the same thing in the same order. Generosity wins the round robin, where the field is full of unadapted opponents and stays that way. Contrition wins the evolutionary run, where the unadapted opponents are gone by generation two hundred and the only thing left to be good at is recovering from your own mistakes. Pavlov finishes between fourth and twelfth depending on the noise level and never survives a single evolutionary run above zero noise.

The cost of noise to the population is the thing worth carrying away. Mean payoff falls from 3.0000 at no noise to 2.6421 at 20%, a loss of 11.9%. That decline happens in a population that is 100% cooperative in intention, made entirely of one forgiving strategy, with no defector anywhere in it. The loss is not caused by anybody choosing to defect. It is the price of the errors themselves plus the price of cleaning up after them.

The Strongest Objection We Could Make

Here is the objection, and it is a good one. The whole result is an artefact of the strategy list, so it measures the list and nothing else.

Take it seriously, because it is substantially true. A round robin score is an average over opponents, so it is arithmetic on a set that somebody chose. We chose ours. We put Joss in, and Joss is the single reason generous tit for tat beats tit for tat at zero noise. We put alternator, always cooperate and random in, and those three are the reason grim trigger and alternator score as well as they do. Remove the four exploiters and the noiseless winner becomes grim trigger. Keep only the eight nice strategies and there is no winner at all: every one of them scores exactly 3.000 against every other, so the tournament ends in an eight-way tie at the maximum and the sort order breaks it arbitrarily. A room with nobody willing to punish anything cannot tell its members apart. Rapoport, Seale and Colman made this case against Axelrod directly and in detail, showing that tit for tat's victory turns on how the tournament is formatted and scored, and on the particular payoff numbers [12]. Our result is one more data point in their column.

So what survives? Three things, and they survive because they do not depend on the field.

The pairwise payoffs are properties of two strategies and the noise, and a payoff matrix cell is the same number whoever else is in the room. The exact Markov chain for two tit-for-tats is field-independent, and the collapse of that pair from 3.0 to 2.46 at 1% noise is not an artefact of anything. The replicator run is field-dependent in its starting mixture but not in its end state: from a uniform start over these fifteen, contrite tit for tat is the dominant survivor at every noise level above zero. And the ordering of tit for tat against its own forgiving and contrite variants under noise is the same ordering Bendor, Kramer and Stout found with a completely different strategy set [6], and the same one Wu and Axelrod found with a third [10]. When three different fields agree, the result is probably about the strategies.

What does not survive is any statement of the form "strategy X is the best strategy". Press and Dyson proved that the iterated prisoner's dilemma contains strategies that can unilaterally set an opponent's score, which puts a formal end to the idea of a single best rule [13], and Stewart and Plotkin showed that within the same family the generous members, not the extortionate ones, are what evolution actually favours [14]. We can say which of fifteen named programs scored highest in one specific room. That is the claim, and it is smaller than the claim people usually make from tournaments like this.

Four choices that would have moved the headline

Four modelling decisions, each defensible, each capable of moving the headline.

Match length. We used 200 rounds because Axelrod did. At 50 rounds tit for tat wins the noiseless tournament outright, and at 20 rounds it slips behind contrite tit for tat while at 500 it slips behind generous tit for tat. Every exploiting strategy gets its free lunch once, at the start, so its advantage is diluted by whatever length we chose. The length is a parameter of the study, and a study that reported only "tit for tat came first" at 50 rounds would be telling the truth and hiding the mechanism.

Scoring the twin match. Axelrod scored every entrant against a copy of itself, so we did too. Drop the twin and at 5% noise the winner stops being contrite tit for tat and becomes alternator. Nice strategies earn 3 against themselves and that self-match props them up; take it away and a strategy that exploits others without needing to get along with itself moves to the front.

Which kind of noise. Ours is misimplementation: the error happens to the move, and both players see it. If we had modelled misperception, where the move is played correctly and only the opponent's copy of it is corrupted, contrite tit for tat would lose its whole advantage. It works by knowing that its own defection was an accident, and under misperception it has no way to know. We would expect generosity to hold the lead at every noise level under that model, and we did not run it.

The field. Already argued above, and the largest of the four. Full field at zero noise: generous tit for tat. Exploiters removed: grim trigger. Free lunches removed: generous tit for tat. Nice strategies only: an exact eight-way tie at 3.000, which is not a winner at all.

Watching an Estimate Settle

Every number above is a Monte Carlo estimate and deserves to be treated like one. The unit of replication is the whole tournament: one repetition means every pair playing one 200-round match, and the standard error is the spread across repetitions divided by the square root of their number.

The left panel is the running mean at 5% noise as repetitions accumulate. The first twenty repetitions are useless. The estimate wanders by several hundredths and the top two swap places twice. By a hundred repetitions the ordering has set and the bands have separated. At four hundred, the leader's 95% interval is 0.00474 wide on each side and the gap it has to resolve is 0.07785, larger by a factor of 16.4. The gap is safe. The zero-noise gap between generous tit for tat and tit for tat, 0.01109 against a paired standard error of 0.00365, is a great deal less safe, which is why we quote its \(t\) statistic rather than just asserting it.

The right panel is a different kind of check, on a parameter rather than on a sample size. Generous tit for tat has one dial, the probability \(g\) of forgiving a defection, and Molander derived an optimum for it [4],

Our scan at 5% noise peaks at \(g = 0.20\) with a score of 2.25562, against 2.24630 at Molander's \(1/3\). The two differ by 1.1 standard errors, so the scan cannot separate them. The curve is flat from about 0.15 to 0.5 and Molander's value sits inside the plateau. The ends are not flat at all. At \(g = 0\), which is plain tit for tat, the score falls to 2.14656 and the rank to fifth. At \(g = 1\), which is always cooperate wearing a costume, it falls to 2.06426 and fourteenth. Forgiveness has a broad optimum and two sharp cliffs, and Molander's algebra lands on the plateau. We are careful to call this agreement suggestive rather than a test, because his derivation is about evolutionary stability between two strategies and ours is a fifteen-way round robin. They are not the same question.

What a Game Can Honestly Say About Living Things

We enjoyed this. Running a tournament is a good time, and watching a strategy everybody has heard of get quietly overtaken by two strategies almost nobody has heard of was the best afternoon the club has had this term. But the last section of an article like this one has to be about what the result is not.

It is not a measurement of cooperation. It is a measurement of a game with two moves, four payoffs, fixed partners, no communication, no reputation, no way to walk away and no way to choose who you play. Axelrod and Hamilton proposed the iterated prisoner's dilemma as a model for reciprocal altruism in biology [3], and it has been enormously productive as a way of organising thought. It has been a good deal less successful as a description of what animals do. Milinski's sticklebacks approaching a predator in what looked like tit for tat [16] were the standard example for years, and the interpretation has been contested ever since, on the grounds that fish doing something that resembles a strategy is not evidence that the strategy is what they are doing. Stephens, McLinn and Stevens got blue jays to sustain reciprocal cooperation only after engineering the payoffs so that the future mattered enough, and the engineering was the finding [17]. Clear cases of reciprocity by the tit-for-tat mechanism in non-human animals remain rare enough to be argued about individually.

Humans do play these games in laboratories, and when experimenters add noise the results rhyme with ours. Fudenberg, Rand and Dreber found that people facing an error-prone repeated prisoner's dilemma converge on strategies that are slow to anger and fast to forgive, and that strict reciprocators do poorly [15]. That is a genuine empirical result about people, and it agrees with the direction of our simulation. It is not produced by our simulation, and our simulation is not evidence for it.

What the computation actually shows is narrower and, we think, more useful. In a system where actions are copied and errors are possible, a rule that reacts to every defection has no way of telling a mistake from an attack, and that single missing distinction is expensive. One error in a thousand moves costs a pair of strict reciprocators 3.3% of everything they would have earned. The two repairs are to react less than fully, which is generosity, or to keep track of who started it, which is contrition. Under our model contrition is better, and it is better by a margin that grows with the error rate until the error rate gets high enough that nothing works. That is a statement about information and repair, and whether it applies to anything with a heartbeat is a question our laptop cannot answer.

The last thing. At 20% noise, always defect won the round robin with 2.25965. In a world where one move in five arrives wrong, the best reply is to stop paying attention to what anyone did and assume the worst. That is not a lesson about cooperation. It is a lesson about signal, and about what happens to every cooperative arrangement when the channel carrying it degrades past a certain point. The strategies did not get worse. The room did.

Reproducing this

One file, one command. Python 3.12 and numpy, nothing else.

python cooperation-tournament.py > cooperation-tournament-output.txt

Expected runtime is about five minutes on a modern laptop. Ours took 289.1 seconds with numpy 2.4.2 on Python 3.12.3, of which about 180 seconds is the ten-level main sweep and the rest is the sensitivity analyses. The master seed is 20250913, hard-coded at the top of the file, and every stage draws an independent stream from it through numpy's SeedSequence.spawn, so the output is deterministic. Setting the environment variable SJC_JSON to a path also writes the figure data as JSON. If your numbers differ from ours by more than the printed standard errors, something is wrong and we would like to hear about it. The interactive model runs a smaller version of both halves in your browser.

References