Evolutionary escape from the prisoner's dilemma

The classic prisoner's dilemma model of game theory is modified by introducing occasional variations on the options available to players. Mutation and selection of game options reliably change the game matrix, gradually, from a prisoner's dilemma game into a byproduct mutualism one, in which cooperation is stable, and "temptation to defect" is replaced by temptation to cooperate. This result suggests that when there are many different potential ways of interacting, exploring those possibilities may make escape from prisoner's dilemmas a common outcome in the world. A consequence is that persistent prisoner's dilemma structures may be less common than one might otherwise expect.     

Teaching and learning in a probabilistic prisoner's dilemma

The prisoner's dilemma is much studied in social psychology and decision-making because it models many real-world conflicts. In everyday terms, the choice to 'cooperate' (maximize reward for the group) or 'defect' (maximize reward for the individual) is often attributed to altruistic or selfish motives. Alternatively, behavior during a dilemma may be understood as a function of reinforcement and punishment. Human participants played a prisoner's-dilemma-type game (for points exchangeable for money) with a computer that employed either a teaching strategy (a probabilistic version of tit-for-tat), in which the computer reinforced or punished participants' cooperation or defection, or a learning strategy (a probabilistic version of Pavlov), in which the computer's responses were reinforced and punished by participants' cooperation and defection. Participants learned to cooperate against both computer strategies. However, in a second experiment which varied the context of the game, they learned to cooperate only against one or other strategy; participants did not learn to cooperate against tit-for-tat when they believed that they were playing against another person; participants did not learn to cooperate against Pavlov when the computer's cooperation probability was signaled by a spinner. The results are consistent with the notion that people are biased not only to cooperate or defect on individual social choices, but also to employ one or other strategy of interaction in a pattern across social choices.     

Evolution of interactions and cooperation in the spatial prisoner's dilemma game

We study the evolution of cooperation in the spatial prisoner's dilemma game where players are allowed to establish new interactions with others. By employing a simple coevolutionary rule entailing only two crucial parameters, we find that different selection criteria for the new interaction partners as well as their number vitally affect the outcome of the game. The resolution of the social dilemma is most probable if the selection favors more successful players and if their maximally attainable number is restricted. While the preferential selection of the best players promotes cooperation irrespective of game parametrization, the optimal number of new interactions depends somewhat on the temptation to defect. Our findings reveal that the "making of new friends" may be an important activity for the successful evolution of cooperation, but also that partners must be selected carefully and their number limited.     

A study on emergence of alternating reciprocity in a 2 x 2 game with 2-length memory strategy

It is recognized that bilateral cooperation (C), a reward-state in other words, emergently comes up in a 2 × 2 prisoner's dilemma game, if you assume a strategy set with a memory concept. Also observed is a mixture state of cooperation (C) and defect (D), saint- and temptation-state in other words, to obtain a higher payoff than R (R reciprocity) in a hero or leader game that is a chicken-type dilemma game; this phenomenon is called alternating reciprocity (AR) or ST reciprocity. Observing a holistic 2 × 2 game world including trivial games and various dilemma games, where 2-length memory and infinite interactions are assumed, the paper reports on the specific mechanism of AR. It is observed there are three different phases relating to AR, which can be explained by the stress of the dilemma.     

The evolution of reciprocity in sizable groups

Recently, several authors have investigated the evolution of reciprocal altruism using the repeated prisoner's dilemma game. These models suggest that natural selection is likely to favor behavioral strategies leading to reciprocal cooperation when pairs of individuals interact repeatedly in potentially cooperative situations. Using the repeated n-person prisoner's dilemma game, we consider whether reciprocal altruism is also likely to evolve when social interactions involve more individuals. We show that the conditions that allow the evolution of reciprocal cooperation become extremely restrictive as group size increases.     

The evolution of payoff matrices: providing incentives to cooperate

Most of the work in evolutionary game theory starts with a model of a social situation that gives rise to a particular payoff matrix and analyses how behaviour evolves through natural selection. Here, we invert this approach and ask, given a model of how individuals behave, how the payoff matrix will evolve through natural selection. In particular, we ask whether a prisoner's dilemma game is stable against invasions by mutant genotypes that alter the payoffs. To answer this question, we develop a two-tiered framework with goal-oriented dynamics at the behavioural time scale and a diploid population genetic model at the evolutionary time scale. Our results are two-fold: first, we show that the prisoner's dilemma is subject to invasions by mutants that provide incentives for cooperation to their partners, and that the resulting game is a coordination game similar to the hawk-dove game. Second, we find that for a large class of mutants and symmetric games, a stable genetic polymorphism will exist in the locus determining the payoff matrix, resulting in a complex pattern of behavioural diversity in the population. Our results highlight the importance of considering the evolution of payoff matrices to understand the evolution of animal social systems.     

Resolving the iterated prisoner's dilemma: theory and reality

Pairs of unrelated individuals face a prisoner's dilemma if cooperation is the best mutual outcome, but each player does best to defect regardless of his partner's behaviour. Although mutual defection is the only evolutionarily stable strategy in one-shot games, cooperative solutions based on reciprocity can emerge in iterated games. Among the most prominent theoretical solutions are the so-called bookkeeping strategies, such as tit-for-tat, where individuals copy their partner's behaviour in the previous round. However, the lack of empirical data conforming to predicted strategies has prompted the suggestion that the iterated prisoner's dilemma (IPD) is neither a useful nor realistic basis for investigating cooperation. Here, we discuss several recent studies where authors have used the IPD framework to interpret their data. We evaluate the validity of their approach and highlight the diversity of proposed solutions. Strategies based on precise accounting are relatively uncommon, perhaps because the full set of assumptions of the IPD model are rarely satisfied. Instead, animals use a diverse array of strategies that apparently promote cooperation, despite the temptation to cheat. These include both positive and negative reciprocity, as well as long-term mutual investments based on 'friendships'. Although there are various gaps in these studies that remain to be filled, we argue that in most cases, individuals could theoretically benefit from cheating and that cooperation cannot therefore be explained with the concept of positive pseudo-reciprocity. We suggest that by incorporating empirical data into the theoretical framework, we may gain fundamental new insights into the evolution of mutual reciprocal investment in nature.     

Cooperation and the scale of competition in humans

Explaining cooperation is one of the greatest challenges for evolutionary biology. It is particularly a problem in species such as humans, where there is cooperation between nonrelatives. Numerous possible solutions have been suggested for the problem of cooperation between nonrelatives, including punishment, policing, and various forms of reciprocity. Here, we suggest that local competition for resources can pose a problem for these hypotheses, analogous to how it can select against cooperation between relatives. We extend the prisoner's dilemma (PD) game to show that local competition between interacting individuals can reduce selection for cooperation between nonrelatives. This is because, with local competition, fitness is relative to social partners, and cooperation benefits social partners. We then test whether nonrelated humans adjust their level of cooperation facultatively in response to the scale of competition when playing the PD for cash prizes. As predicted, we found that individuals were less likely to cooperate when competition was relatively local. Cooperation between humans will therefore be most likely when repeated interactions take place on a local scale between small numbers of people, and competition for resources takes place on a more global scale among large numbers of people.     

Game dynamics and evolutionary transitions

An evolutionary model based on the Taylor-Jonker game dynamics is presented. A set of strategies is compatible if there exists a dynamical equilibrium between its members and there is an evolutionary transition to another compatible set if new mutant strategies bring about a passage to another equilibrium. We apply these concepts to supergame strategies, which play repeatedly a given matrix game and at each time step choose their pure strategy according to the preceding moves of the opponent. We investigate the patterns of evolution in zero-sum games, games of partnership, the prisoner's dilemma and the hawkdove game.     

Tit-for-tat or win-stay, lose-shift?

The repeated Prisoner's Dilemma is usually known as a story of tit-for-tat (TFT). This remarkable strategy has won both of Robert Axelrod's tournaments. TFT does whatever the opponent has done in the previous round. It will cooperate if the opponent has cooperated, and it will defect if the opponent has defected. But TFT has two weaknesses: (i) it cannot correct mistakes (erroneous moves) and (ii) a population of TFT players is undermined by random drift when mutant strategies appear which play always-cooperate (ALLC). Another equally simple strategy called 'win-stay, lose-shift' (WSLS) has neither of these two disadvantages. WSLS repeats the previous move if the resulting payoff has met its aspiration level and changes otherwise. Here, we use a novel approach of stochastic evolutionary game dynamics in finite populations to study mutation-selection dynamics in the presence of erroneous moves. We compare four strategies: always-defect (ALLD), ALLC, TFT and WSLS. There are two possible outcomes: if the benefit of cooperation is below a critical value then ALLD is selected; if the benefit of cooperation is above this critical value then WSLS is selected. TFT is never selected in this evolutionary process, but lowers the selection threshold for WSLS.     

What initially brought about communications?

This paper reports an intelligent agent equipped with two-layer finite state machines (FSMs) that can communicate by turning lighting on and off, leading to social cooperation that solves the dilemma situation, modeled by a one-shot 2x2 game. This communication between two gaming agents can be observed in hero- and leader-type dilemma games, where alternating reciprocity, repeating cooperation (C)-defeat (D) after D-C, is the equal pareto optimum instead of a sequence of mutual cooperation that is the equal pareto optimum for a prisoner's dilemma (PD) game.     

Invasion and expansion of cooperators in lattice populations: Prisoner's dilemma vs. snowdrift games

The evolution of cooperation is an enduring conundrum in biology and the social sciences. Two social dilemmas, the prisoner's dilemma and the snowdrift game have emerged as the most promising mathematical metaphors to study cooperation. Spatial structure with limited local interactions has long been identified as a potent promoter of cooperation in the prisoner's dilemma but in the spatial snowdrift game, space may actually enhance or inhibit cooperation. Here we investigate and link the microscopic interaction between individuals to the characteristics of the emerging macroscopic patterns generated by the spatial invasion process of cooperators in a world of defectors. In our simulations, individuals are located on a square lattice with Moore neighborhood and update their strategies by probabilistically imitating the strategies of better performing neighbors. Under sufficiently benign conditions, cooperators can survive in both games. After rapid local equilibration, cooperators expand quadratically until global saturation is reached. Under favorable conditions, cooperators expand as a large contiguous cluster in both games with minor differences concerning the shape of embedded defectors. Under less favorable conditions, however, distinct differences arise. In the prisoner's dilemma, cooperators break up into isolated, compact clusters. The compact clustering reduces exploitation and leads to positive assortment, such that cooperators interact more frequently with other cooperators than with defectors. In contrast, in the snowdrift game, cooperators form small, dendritic clusters, which results in negative assortment and cooperators interact more frequently with defectors than with other cooperators. In order to characterize and quantify the emerging spatial patterns, we introduce a measure for the cluster shape and demonstrate that the macroscopic patterns can be used to determine the characteristics of the underlying microscopic interactions.     

Coveting thy neighbors fitness as a means to resolve social dilemmas

In spatial evolutionary games the fitness of each individual is traditionally determined by the payoffs it obtains upon playing the game with its neighbors. Since defection yields the highest individual benefits, the outlook for cooperators is gloomy. While network reciprocity promotes collaborative efforts, chances of averting the impending social decline are slim if the temptation to defect is strong. It is, therefore, of interest to identify viable mechanisms that provide additional support for the evolution of cooperation. Inspired by the fact that the environment may be just as important as inheritance for individual development, we introduce a simple switch that allows a player to either keep its original payoff or use the average payoff of all its neighbors. Depending on which payoff is higher, the influence of either option can be tuned by means of a single parameter. We show that, in general, taking into account the environment promotes cooperation. Yet coveting the fitness of one's neighbors too strongly is not optimal. In fact, cooperation thrives best only if the influence of payoffs obtained in the traditional way is equal to that of the average payoff of the neighborhood. We present results for the prisoner's dilemma and the snowdrift game, for different levels of uncertainty governing the strategy adoption process, and for different neighborhood sizes. Our approach outlines a viable route to increased levels of cooperative behavior in structured populations, but one that requires a thoughtful implementation.     

Efficiency in evolutionary games: Darwin, Nash and the secret handshake

This paper considers any evolutionary game possessing several evolutionarily stable strategies, or ESSs, with differing payoffs. A mutant is introduced which will "destroy" any ESS which yields a lower payoff than another. This mutant possesses a costless signal and also conditions on the presence of this signal in each opponent. The mutant then can protect itself against a population playing an inefficient ESS by matching this against these non-signalers. At the same time, the mutants can achieve the more efficient ESS against the signaling mutant population itself. This construction is illustrated by means of the simplest possible example, a co-ordination game. The one-shot prisoner's dilemma is used to illustrate how a superior outcome which is not induced by an ESS may be temporarily but not permanently attained. In the case of the repeated prisoner's dilemma, the present argument seems to render the "evolution of co-operation" ultimately inevitable.     

The continuous prisoner's dilemma and the evolution of cooperation through reciprocal altruism with variable investment

Understanding the evolutionary origin and persistence of cooperative behavior is a fundamental biological problem. The standard "prisoner's dilemma," which is the most widely adopted framework for studying the evolution of cooperation through reciprocal altruism between unrelated individuals, does not allow for varying degrees of cooperation. Here we study the continuous iterated prisoner's dilemma, in which cooperative investments can vary continuously in each round. This game has been previously considered for a class of reactive strategies in which current investments are based on the partner's previous investment. In the standard iterated prisoner's dilemma, such strategies are inferior to strategies that take into account both players' previous moves, as is exemplified by the evolutionary dominance of "Pavlov" over "tit for tat." Consequently, we extend the analysis of the continuous prisoner's dilemma to a class of strategies in which current investments depend on previous payoffs and, hence, on both players' previous investments. We show, both analytically and by simulation, that payoff-based strategies, which embody the intuitively appealing idea that individuals invest more in cooperative interactions when they profit from these interactions, provide a natural explanation for the gradual evolution of cooperation from an initially noncooperative state and for the maintenance of cooperation thereafter.     

Replicator dynamics for optional public good games

The public goods game represents a straightforward generalization of the prisoner's dilemma to an arbitrary number of players. Since the dominant strategy is to defect, both classical and evolutionary game theory predict the asocial outcome that no player contributes to the public goods. In contrast to the compulsory public goods game, optional participation provides a natural way to avoid deadlocks in the state of mutual defection. The three resulting strategies--collaboration or defection in the public goods game, as well as not joining at all--are studied by means of a replicator dynamics, which can be completely analysed in spite of the fact that the payoff terms are nonlinear. If cooperation is valuable enough, the dynamics exhibits a rock-scissors-paper type of cycling between the three strategies, leading to sizeable average levels of cooperation in the population. Thus, voluntary participation makes cooperation feasible. But for each strategy, the average payoff value remains equal to the earnings of those not participating in the public goods game.     

Beyond the prisoner's dilemma: Toward models to discriminate among mechanisms of cooperation in nature

The iterated prisoner's dilemma game, or IPD, has now established itself as the orthodox paradigm for theoretical investigations of the evolution of cooperation; but its scope is restricted to reciprocity, which is only one of three categories of cooperation among unrelated individuals. Even within that category, a cooperative encounter has in general three phases, and the IPD has nothing to say about two of them. To distinguish among mechanisms of cooperation in nature, future theoretical work on the evolution of cooperation must distance itself from economics and develop games as a refinement of ethology's comparative approach.     

Long-term social bonds promote cooperation in the iterated Prisoner's Dilemma

Reciprocal altruism, one of the most probable explanations for cooperation among non-kin, has been modelled as a Prisoner''s Dilemma. According to this game, cooperation could evolve when individuals, who expect to play again, use conditional strategies like tit-for-tat or Pavlov. There is evidence that humans use such strategies to achieve mutual cooperation, but most controlled experiments with non-human animals have failed to find cooperation. One reason for this could be that subjects fail to cooperate because they behave as if they were to play only once. To assess this hypothesis, we conducted an experiment with monogamous zebra finches (Taeniopygia guttata) that were tested in a two-choice apparatus, with either their social partner or an experimental opponent of the opposite sex. We found that zebra finches maintained high levels of cooperation in an iterated Prisoner''s Dilemma game only when interacting with their social partner. Although other mechanisms may have contributed to the observed difference between the two treatments, our results support the hypothesis that animals do not systematically give in to the short-term temptation of cheating when long-term benefits exist. Thus, our findings contradict the commonly accepted idea that reciprocal altruism will be rare in non-human animals.     

Revisiting “The Evolution of Reciprocity in Sizable Groups”: Continuous reciprocity in the repeated n-person prisoner's dilemma

For many years in evolutionary science, the consensus view has been that while reciprocal altruism can evolve in dyadic interactions, it is unlikely to evolve in sizable groups. This view had been based on studies which have assumed cooperation to be discrete rather than continuous (i.e., individuals can either fully cooperate or else fully defect, but they cannot continuously vary their level of cooperation). In real world cooperation, however, cooperation is often continuous. In this paper, we re-examine the evolution of reciprocity in sizable groups by presenting a model of the n-person prisoner's dilemma that assumes continuous rather than discrete cooperation. This model shows that continuous reciprocity has a dramatically wider basin of attraction than discrete reciprocity, and that this basin's size increases with efficiency of cooperation (marginal per capita return). Further, we find that assortative interaction interacts synergistically with continuous reciprocity to a much greater extent than it does with discrete reciprocity. These results suggest that previous models may have underestimated reciprocity's adaptiveness in groups. However, we also find that the invasion of continuous reciprocators into a population of unconditional defectors becomes realistic only within a narrow parameter space in which the efficiency of cooperation is close to its maximum bound. Therefore our model suggests that continuous reciprocity can evolve in large groups more easily than discrete reciprocity only under unusual circumstances.     

A Theoretical Analysis of Temporal Difference Learning in the Iterated Prisoner’s Dilemma Game

Direct reciprocity is a chief mechanism of mutual cooperation in social dilemma. Agents cooperate if future interactions with the same opponents are highly likely. Direct reciprocity has been explored mostly by evolutionary game theory based on natural selection. Our daily experience tells, however, that real social agents including humans learn to cooperate based on experience. In this paper, we analyze a reinforcement learning model called temporal difference learning and study its performance in the iterated Prisoner’s Dilemma game. Temporal difference learning is unique among a variety of learning models in that it inherently aims at increasing future payoffs, not immediate ones. It also has a neural basis. We analytically and numerically show that learners with only two internal states properly learn to cooperate with retaliatory players and to defect against unconditional cooperators and defectors. Four-state learners are more capable of achieving a high payoff against various opponents. Moreover, we numerically show that four-state learners can learn to establish mutual cooperation for sufficiently small learning rates.