## Monday, 20 December 2010

### Iterated Prisoner's Dilemma

Last Sunday we had a sort of workshop on "negotiation" for some of the Mozilla managers. Part of this workshop was an exercise amounting to playing Iterated Prisoner's Dilemma, with some twists which aren't relevant here. There were three independent games, each with two players, each player being a team of three or four people. We were instructed to maximise our own score without regard to the other player's score.

My approach was to observe that if both teams behave perfectly rationally, they will use the same strategy. Then the optimal strategy is obviously to always cooperate. If the other team defects then you know the rationality assumption doesn't hold, so the game gets more difficult (probably devolving to the game-theoretic equilibrium of always defecting, but then asymptotically it doesn't matter what you did before detecting defection). Fortunately for us the other team was in fact rational enough to always cooperate and we played a perfect game. (If the number of iterations is known in advance, defecting on or before the last round might make sense, but we couldn't be sure how many rounds there were going to be so this didn't matter.) This approach is known as superrationality although I wasn't aware of that at the time...

One surprise for me was that at least three of the six teams did not always cooperate. I was a bit disappointed to find that not all my colleagues think like me :-). I don't think that was supposed to be the lesson of this exercise, though :-).

Another surprise was that some people expected me to play the game according to "Christian principles". In fact, I was purely trying to optimize the objective function we were given, regardless of what God would want. This raises a very interesting question: is it OK to disobey God's commands within the framework of an artificial game? In many games, especially games with more than two players that involve alliances, lies and betrayal are an essential part of the game. Is it OK for a Christian to lie and betray in such games?

I think the answer depends on the context. If everyone understand it's "just a game", the diabolical behaviour is demanded by the framework of the game, and there are absolutely no hard feelings about such behaviour, it seems harmless. But in any context where others might expect you to be obeying God even within the framework of the game, I think we have to follow Paul and play it safe so we do not "cause others to stumble". So I think in hindsight I was wrong to ignore Christian considerations in our negotiation exercise, although I think they lead to the same results anyway.

So what would Jesus do in a real-life Iterated Prisoner's Dilemma? It seems obvious: "do to others what you would have them do to you", which leads to the same analysis as I had before --- cooperate. If the other player defects, the Sermon on the Mount still applies --- "turn the other cheek --- and also "forgive seventy times seven times": we are asked to cooperate in the face of repeated defection.

Interestingly, in this game, belief in the trustworthiness of the other player --- or even belief in divine justice --- lead to superior joint outcomes on average even if the belief is false. Most people think it's obvious we should believe only what is true, but I don't think that principle is obvious at all in (for example) materialist utilitarian frameworks. It should be obvious to Christians.

1. You might find this article from Discover Magazine interesting. I read it when it hit the shelves, and I've never forgotten it.
http://discovermagazine.com/1993/may/forgivenessmath212

2. Anyone expecting you to uphold real-world moral principles in a *game* (apart from meta-game princples like not cheating) has some serious problems distinguishing fantasy from reality.

3. Interesting. What strategy were you actually following? Always-cooperate or tit-for-tat?

4. Perhaps the rules you were playing by were different than the classic iterated prisoners' dilemma, but turn the other cheek is a recipe for ruin. I thought that tit for tat was the winning strategy on average -- you start by giving the other guy the benefit of the doubt, but after that, you always echo his previous response.

5. Robert O'Callahan20 December 2010 at 23:14

Turn the other cheek is a recipe for ruin if it's interpreted as "always cooperate" AND optimizing your own payoff is really all that matters. Neither of those necessarily hold in the real world.
If you read Alex's link (or Wikipedia) you'll see that "Generous Tit For Tat" can actually beat Tit For Tat if you model populations.
db48x: We hadn't completely worked out what to do if the other player defected, since it didn't happen --- our secondary goal was "not think more than necessary" :-). But I'm pretty sure we would have retaliated, so we weren't playing "always cooperate".
Anonymous: right, I was surprised by this, but in retrospect I shouldn't have been. It's easy for emotions to leak across the game boundary. I'm sure we've all experienced this; I certainly have needed to be reminded "it's just a game!" when I'm grumpy about losing!

6. I've seen this strange topic, religion and iterated prisonner's dilemmas, discussed at less wrong recently. The conclusion: not being rational can be a useful advantage.
http://lesswrong.com/lw/32u/diplomacy_as_a_game_theory_laboratory/

7. Haven't gone through the links (sorry) but there is also the idea of "tit for tat" being Old Testament and "always co-operate" being New Testament.
Application of morality in games is an interesting question. Taking the example of soccer - in a friendly social/mixed sex league, a bit of jostling may be taken as not "part of the game". In a competitive league, jostling and physicality may be OK, but diving and abusing the ref are not. In some parts of the world, pretty much anything is fair game, until the final whistle blows.
The question of what is "part of the game", and what we should instead apply non-game morality to is very context sensitive.
Then you continue this line of reasoning to people who say that work or [insert non-life-or-death activity here] is "just a game" etc etc...

8. As an aside I'd imagine most of the readers of this blog are au fait with game theory incl of the evolutionary sort but FWIW a fascinating (introductory & not particularly technical) book on the subject is Brian Skryms "Evolution of the Social Contract"
http://www.amazon.com/Evolution-Social-Contract-Brian-Skyrms/dp/0521555833/

9. My study of the Bible indicates to me that the old testament presents the idea of "up to tit for tat", and even the new testament "turn the other cheek" that sounds like "always cooperate" is explained in another place as a form of revenge. "pouring coals of fire on someone's head"
That brings me to my other observation. The game is rather artificially limited in its outcomes, and does not account for the reality that has often turned up in such cases in real life. Many such cases came up during world war 2 for members of the underground/resistance movements. Even though the Gestapo and other police forces tried to set the score at approximately what this game suggests, the reality always gave more options to the prisoners.

10. The "virtual world" scenario is similar. My son plays Civilization and Age of Empires. Sometimes, we'll run a scenario where he invest entirely in religion/priests, attempting to conquer the enemy through conversion. Generally, he chooses the best path given the recommendations in the game.
For instance, "researching" nationalism and fascism leads to an incredibly cohesive and powerful state. We know this in real life from the rise of Germany post WWI. Yet, we also know the consequences of that decision. Should he choose pacifism or meditation instead, even in the game? If we're playing Monopoly, and someone asks for money, should we give it to them? Shouldn't we all just put all our money in the pot and let one of the players dole it out to each as s/he has need?
Also, even in, say, poker (with no betting), the loser will be upset that they lost, so there are always repercussions.
Random thoughts.

11. explained in another place as a form of revenge. "pouring coals of fire on someone's head"
While this reading comports with the dictionary-based meaning of Proverbs 25:21-22, in the overall context provided by the Bible I don't believe it is the most sensible interpretation of the passage. Your enemies will be shown the error of their ways, but this is not a good because it is a way to "get back" at them. Rather, it is a good because it is an incentive to cause them to do good rather than evil. Their shame is not of your doing but rather of their actions. As the Bible says in Romans 12:19, quoting the Old Testament's Deuteronomy 32:35, revenge is to be left to God. Now it is of course true that one may do right, but for wrong motives. But the problem in such cases is not with doing right -- that you must do regardless -- but with what you have striven to achieve.

12. Good exercise... I should try it with my employees too at the next training or team building. What they played at the last one amused them but they thought everything was too easy, like playing Jewel Quest Solitaire or something. And the idea of "game theory" is definitely something I must investigate further.. I'm sort of new in the business field, got a lot of things to learn about how to invest in my team.